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Report  on  Distance  Learning  Technologies 


Abstract:  This  report  provides  a  wide  view  of  the  costs,  risks,  and  benefits 
associated  with  instructional  technology  alternatives.  The  number  and  variety 
of  possible  paths  to  learning  through  this  technology  have  increased  markedly 
in  recent  years  with  the  advent  of  interactive  multimedia,  satellite 
communications,  and  the  Internet.  More  than  ever  before,  learning 
technologies  have  created  new  educational  possibilities  for  people  and 
organizations.  In  this  era  of  increasing  financial  stringency,  we  are  obligated  to 
examine  these  new  possibilities  in  light  of  their  obvious  advantages:  replication 
of  high-quality  instruction,  lower  overall  costs,  increased  quality  in  educational 
outcomes,  and  the  ability  to  provide  these  benefits  over  long  distances.  This 
report  will  show  that  with  today’s  computer-based  instructional  technology,  the 
question  is  no  longer  whether  to  use  the  technology,  but  rather  howto  use  it. 


1  Overview 

This  report  provides  a  technology-independent  rationale  for  an  overall  distance-learning  strat¬ 
egy,  including  a  compilation  of  the  author’s  and  reviewers’  views  on  distance-learning  ap¬ 
proaches. 

Following  an  introduction,  overview,  statement  of  the  objectives  of  the  report,  and  a  presenta¬ 
tion  of  the  assumptions  guiding  the  report.  Section  1 .3,  Current  Status,  describes  the  current 
educational  and  technological  decision  space  that  this  document  addresses.  Section  2,  A 
High-Level  Look  at  Core  Educational  Issues,  is  a  discussion  of  features  that  affect  learning. 
My  intention  in  writing  this  chapter  is  to  provide  the  reader  with  a  description  of  certain  educa¬ 
tional  premises  used  in  making  determinations  about  the  relative  effectiveness  of  learning  sys¬ 
tems.  Section  3,  Computer-Based  Multimedia  and  Related  Solutions,  summarizes  the 
characteristics  of  the  set  of  educational  technologies  described  in  this  report.  A  large  part  of 
this  section  deals  with  multimedia  and  related  solutions,  because  multimedia  represents  ex¬ 
tremes  on  two  fronts:  tremendous  positive  potential  for  distance  learning  and  high  develop¬ 
ment  cost.  Additionally,  multimedia  and  related  solutions  include  a  superset  of  the  issues  for 
all  of  the  computer-based  learning  systems  that  could  possibly  be  deployed.  Section  4,  A 
Technology-Specific  Analysis,  is  a  detailed  review  of  these  technologies  relative  to  their  asso¬ 
ciated  management  complexity,  cost,  and  learning-effectiveness  values.  Section  5,  Summary, 
Recommendations,  and  Conclusions,  suggests  possible  deployment  scenarios.  Because  the 
list  could  be  voluminous,  only  a  few  examples  are  given  as  realistic  candidates.  Appendix  A 
provides  one  example  of  the  needs  of  a  course  devleopment  process. 

1 .1  Objective  of  the  Report 

The  purpose  of  this  report  is  to  provide  sufficient  background  knowledge  of  delivery  systems 
to  enable  sound  decisions  to  be  made  about  the  deployment  of  courses  focusing  on  three  lev- 
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els  of  student  proficiency.  I  present  the  systems  in  this  report  in  consideration  of  their  ease  of 
implementation,  cost,  educational  benefit,  and  ease-of-fit  to  today’s  practices. 


1.2  Assumptions 

Alongside  cost  and  risk,  I  will  feature  educational  concerns  prominently  among  the  issues  that 
determine  the  type  of  delivery  solution  recommended.  Assumptions  about  the  organizations 
to  which  this  report  is  addressed  are 

•  A  certain  amount  of  instructional  infrastructure  is  already  in  place. 

-  Satellite  hardware  and  system  support  is  readily  available  for  wide-scale 
transmission  of  one-way  and  potentially  two-way  educational  offerings. 

-The  existing  computer  hardware  base  is  composed  largely  of  Intel® 
microprocessor-based  machines,  capable  of  delivering  text-based 
instruction  [computer-based  training  (CBT)]. 

•  The  organization  is  willing  to  explore  the  possibility  of  expanding  the  existing 
infrastructure  to  include  technologies  capable  of  interactive  multimedia^  and 
other  advanced  computer-based  instructional  solutions. 

1.3  Current  Status 

Given  today’s  austere  budgets  and  the  likelihood  of  future  budget  reductions,  organizations 
need  to  plan  for  this  trend  for  the  foreseeable  future.  At,  say,  $1400  for  travel  and  per  diem  for 
each  student,  the  cost  of  training  can  become  prohibitively  expensive.  Travel  and  per  diem 
alone  often  account  for  more  than  half  of  the  cost  of  a  course.  For  many  organizations,  reduc¬ 
ing  these  travel  and  per  diem  figures  has  become  imperative. 


Multimedia  is  defined  here  as  digital  audio,  video,  images,  music,  text,  or  animation  that  is  integrated  on  a  com- 
puter  system. 
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2  A  High-Level  Look  at  Core  Educational  Issues 


There  are  many  possible  strategies  for  distance  learning  delivery  that  could  support  or  poten¬ 
tially  replace  the  current  state  of  educational  practice.  The  main  issues  are  educational  bene¬ 
fits  and  costs.  The  costs  of  implementing  any  instructional  system,  whether  computer  based 
or  human,  are  high.  There  will  always  be  a  cost  for  developing  a  system,  maintaining  it,  tran¬ 
sitioning  it  to  others  who  will  teach  it,  distributing  it,  and  so  on.  In  this  part  of  the  report,  I  dis¬ 
cuss  costs  and  educational  benefits  in  sufficient  detail  to  enable  a  sound  choice  to  be  made 
among  possible  system  implementations. 

2.1  General  Learning  Factors 

The  best  educational  practices  must  be  folded  into  any  distance-learning  strategy.  While  there 
are  many  other  learning  factors,  the  scope  of  this  report  will  be  limited  to  an  essential  few. 

2.1.1  Interactivity 

Interactivity  is  a  set  of  measurable  instructional  qualities  such  as  “time  on  task,”  “immediacy 
of  feedback,”  “learner  engagement,”  and  so  on.  Laurel  [Laurel  86]  provides  the  following  def¬ 
inition: 

Interactive  frequency,  range,  and  significance  correspond  to  how  often  user 
input  is  enabled,  the  range  of  choices  available  to  users  at  a  given  moment  in 
the  interaction,  and  the  impact  of  the  user’s  choices  and  actions  on  the  whole 
experience. 

In  the  world  of  educational  technology,  interactivity  has  become  a  byword  for  high  quality  in 
learning  materials.  Even  though  this  term  is  still  gaining  clearer  definition  and  is  sometimes 
carelessly  applied,  it  is  useful  in  describing  a  general  characteristic  important  to  any  effective 
instructional  system — ^that  the  learner  must  be  continuously  and  actively  engaged. 

2.1.2  Learner  Adaptability 

Another  area  of  prime  instructional  concern  is  the  adaptability  of  the  learning  system  to  the 
individual  student.  In  human  terms,  this  adaptability  comes  in  the  form  of  a  tutor.  Human  tu¬ 
toring  is  considered  by  educational  researchers  to  be  among  the  most  effective  forms  of  in¬ 
struction  [Anderson  85].  One  reason  for  this  is  the  tutor’s  ability  to  home  in  on  students’ 
misconceptions  and  provide  problems  and  explanations  appropriate  to  individual  needs  in  real 
time. 

Computer  systems  that  emulate  the  methods  of  human  tutors  are  called  intelligent  tutoring 
systems.  These  systems  are  capable  of  providing  all  of  the  interactive  features  of  multimedia 
in  addition  to  precise  commentary  on  the  student’s  performance  and  the  selection  of  remedi¬ 
ations  that  are  appropriate  to  an  individual  student’s  needs. 
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2.1.3  Situated  Learning 

It  is  well  known  that  learning  is  enhanced  when  it  takes  place  in  a  context  that  is  much  like  that 
in  which  the  student  will  be  expected  to  perform  after  instruction.^  While  this  may  seem  obvi¬ 
ous  to  most  of  us,  the  idea  of  situated  learning  has  not  traditionally  been  used  in  most  class¬ 
room  instruction.  The  most  readily  available  example  of  a  situated  learning  experience  is  a 
simulation.  Flight  simulators,  for  example,  are  designed  to  provide  the  trainee  with  as  close  an 
approximation  of  actual  flight  as  possible.  In  theory,  with  enough  refinements,  it  would  be  pos¬ 
sible  to  create  an  instructional  system  that  would  permit  the  immediate  assignment  of  pilots  to 
aircraft  upon  completion  of  instruction.  While  actual  flight  training  always  includes  time  in  a  real 
aircraft,  it  is  technologically  feasible  to  avoid  this  step  entirely. 

In  terms  of  regular  classroom  circumstances,  however,  situated  approaches  call  for  ways  of 
mimicking  the  circumstance  in  which  the  students  will  be  expected  to  use  their  knowledge. 
This  mimicking  can  involve  role  plays  or  any  other  teaching  device  that  will  reconstruct  an  ex¬ 
perience  similar  to  “real  life.”  Moot  court,  simulation,  and  role  play  are  all  common  teaching 
methods  for  situating  the  learning  experience. 

2.1 .4  Retention 

Retention  of  information  is  the  ability  to  recall  it  on  demand  after  instruction.  Anything  learned 
decays  in  the  time  between  the  training  and  application.  Recently,  technology  has  opened  an¬ 
other  door  to  the  possibility  of  providing  these  features  in  the  form  of  on-the-job  electronic  per¬ 
formance  support.  Electronic  performance  support  systems  (EPSS)^  provide  in-context 
instruction  as  a  job  is  being  performed  (see  Section  3.5.1).  The  more  practice  a  student  has 
in  a  realistic  setting,  the  more  likely  the  knowledge  is  to  “stick.”  An  optimal  instructional  system 
would  provide  highly  interactive,  realistic  (situated)  training,  preferably  in  a  time  frame  very 
close  to  its  actual  use  in  a  job  or  on  a  project.  Further,  the  optimal  training  system  would  be 
readily  accessible,  so  that  any  forgotten  information  or  skill  training  could  be  revisited  on  de¬ 
mand. 

2.2  The  Importance  of  Bloom’s  Levels 

Bloom’s  taxonomy  [Bloom  56]  provides  a  means  of  targeting  course  expectations  to  potential 
audiences  (GS  levels).  In  this  section,  I  address  Bloom’s  levels  in  the  context  of  the  technol¬ 
ogies  described  in  this  report. 

There  are  two  primary  assertions  that  provide  useful  guidance  in  applying  Bloom  in  the  tech¬ 
nological  context: 


Beginning  with  the  work  of  David  Ausubel  in  the  mid-1970s,  educational  researchers  began  to  examine  the 
concept  of  grounded  theory,  the  notion  that  it  is  important  to  consider  the  instructional  context  as  a  factor  in 
learning. 

^  For  a  complete  military  case  history  on  EPSS,  see  DTIC  Document  No.  AD-A201  401 ,  Implementing  Embed¬ 
ded  Training  (ET)  [DTIC  88]. 
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1 .  Computer-based  multimedia  can  be  employed  for  any  of  the  six  Bloom’s  lev¬ 
els  of  learning. 

2.  By  properly  specifying  the  objectives  for  a  course  using  Mager’s  model  [Mag- 
er  62]  for  performance  objectives  and  Bloom’s  levels,  reasonably  accurate 
projections  about  the  scope,  scheduling,  and  cost  for  a  multimedia  project 
can  be  determined. 

The  second  assertion  above  is  essential  from  a  project-management  standpoint,  because  the 
complexity  of  the  performance  objectives  will  have  implications  for  the  technical  development 
of  the  system.  For  every  interaction  specified  in  the  curriculum,  there  will  be  a  corresponding 
set  of  system  requirements  implied.  The  first  hints  at  the  overall  complexity  and  scope  of  a 
computer-based  multimedia  system  will  be  seen  through  its  objectives,  and,  depending  on 
Bloom’s  levels,  will  imply  more  or  less  complexity.  For  example,  after  creating  a  series  of  train¬ 
ing  objectives  for  Bloom’s  level  4  through  6,  based  on  their  descriptions,  system  designers  can 
ascertain  some  of  the  complexity  of  interactions  suggested:  how  much  programming  will  be 
needed,  what  kinds  of  photos  and  renderings  may  be  required,  whether  video  will  be  needed, 
and  so  on.  This  level  analysis  is  still  “gut  feel;’’  however,  combined  with  critical  path  analysis 
techniques,  this  kind  of  estimate  is  much  more  meaningful  than  guesswork  alone. 

2.3  Consistency  of  Training 

Consistency  of  training  means  that  once  the  developer  has  managed  to  create  a  successful 
training  system,  there  is  no  degradation  from  one  class  to  another  in  terms  of  instructional 
quality.  The  reason  for  a  tight  specification  of  the  instructional  system  is  to  ensure  consistency 
and  high  quality  in  a  curriculum  used  by  many  instructors.  When  the  objectives  are  spelled  out, 
the  quality  of  instruction  can  be  measured  in  terms  of  the  objectives.  Even  with  tight  specifica¬ 
tion,  there  is  often  much  inconsistency  between  any  two  human  instructors.  One  of  the  positive 
features  of  a  well-designed,  interactive,  computer-based  training  system  is  consistency  of 
training  throughout  the  life  of  the  subject  matter. 

2.4  Costs:  Time  and  Money 

With  technology-based  solutions,  costs  always  depend  on  the  complexity  of  the  solution.  For 
example,  while  a  satellite-based  course  need  not  require  any  more  time  in  design  and  mate¬ 
rials  than  a  regular  on-site  course,  it  will  incur  cost  for  the  uplink  and  added  complexity  for  mail¬ 
ing  of  materials  and  remote  monitoring  of  test  results.  Although  these  processes  can  be  made 
efficient,  costs  related  to  satellite  course  delivery  will  be  relatively  permanent.  With  multimedia, 
there  are  large  costs  for  planning  and  development.  Lead  times  are  often  long  (greater  than 
eight  months  to  delivery),  and  supervision  of  the  development  effort  is  a  required  and  time- 
consuming  activity.  With  traditional  human-based  instruction,  development  and  delivery  costs 
are  high,  especially  with  necessary  travel  included. 
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3  Computer-Based  Multimedia  and  Related  Solutions 


The  following  paragraphs  describe  available  technologies  that  are  candidates  as  distance¬ 
learning  solutions.  I  will  present  a  high-level  view  of  instructional  technologies  as  background 
for  a  more  detailed  comparison  of  each  technology  against  specific  criteria  related  to  develop¬ 
ment  and  learning  factors. 

The  word  multimedia  has  come  to  include  many  possible  educational  delivery  strategies.  In 
this  document,  multimedia  is  the  integration  of  digital  video,  audio,  text,  graphics,  or  sound  (in 
any  combination)  in  a  computer  system.  Virtually  all  computer-based  training  and  educational 
delivery  solutions  could  have  multimedia  elements  associated  with  them.  In  the  most  elemen¬ 
tary  computer-based  training  (CBT)  systems,  the  level  of  interactivity  between  the  user  and 
computer  typically  consists  of  a  set  of  fixed  responses.  A  reasonable  system  response  is  “Sor¬ 
ry,  try  again,”  at  which  point  the  student  is  routed  back  through  a  presentation  of  the  initial 
course  materials  in  the  hope  that  learning  will  happen  on  a  second  or  third  explanation  of  the 
same  concept.  Even  in  the  worst  case,  with  only  text  and  the  limitation  of  fixed  responses,  this 
basic  model  of  instruction  has  proven  to  be  effective.  With  the  addition  of  interactive  videodisc 
and  graphics,  systems  of  this  kind  advanced  in  their  variety  and  diversity,  and  the  term  multi- 
media  began  to  gain  prominence. 

The  following  list  includes  broad  categories  of  the  computer-based  delivery  systems  currently 
available  in  which  some  level  of  multimedia  is  likely  to  be  a  part: 

•  Hypermedia  and  Internet-based  instruction 

•  Just-in-time  lecture 

•  Intelligent  tutoring 

•  Commercial  off-the-shelf  (COTS)  training 

Multimedia  has  stepped  beyond  simple  text-based  responses  such  as  “sorry,  try  again,"  not 
by  changing  the  complexity  of  the  underlying  computer  program,  but  by  adding  color  and  va¬ 
riety  to  the  user’s  environment.  Thanks  to  significant  and  timely  technological  improvements, 
we  now  have  a  wide  assortment  of  media  techniques  that  can  be  applied  to  make  the  learning 
environment  more  interesting  and  dynamic.  In  truth,  there  is  nothing  inherently  new  about  the 
modes  of  delivery  in  the  multimedia-based  learning  technologies.  In  one  form  or  another, 
classroom  teachers  have  been  using  multimedia  since  the  1950s.  Film,  overhead  projectors, 
chalkboards  and  tape  recorders  are  all  forms  of  multimedia  that  teachers  have  used  to  provide 
a  richer  classroom  experience  for  their  students.  All  of  these  media  strategies  now  have  their 
corollaries  in  the  current  multimedia  gold  rush. 

What  is  new  with  computer-based  multimedia  is  the  integration  and  automation  of  these  fea¬ 
tures  in  a  tightly  focused  way,  assuring  repeatability  of  instruction.  Approximately  10  years 
ago,  there  were  many  start-up  efforts  by  researchers  in  the  area  of  videodisc-based  multime¬ 
dia,  beginning  with  such  projects  as  MIT’s  Aspen  Project,  a  system  that  provided  an  interactive 
video  tour  of  Aspen  Colorado.  And  there  were  the  new  Discovision™  systems  with  “level  two” 
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interactivity,  and  the  “Puzzle  of  the  Tacoma  Narrows  Bridge  Collapse,”  which  provided  learn¬ 
ers  with  a  new  and  interesting  window  into  the  physics  of  standing  waves. 

One  important  attribute  that  has  changed  since  that  time  is  the  digitizing  of  media  (particularly 
video)  so  that  we  no  longer  need  to  use  the  large-platter  videodisc  to  capture  analog  video. 
The  change  to  all-digital  media  now  permits  the  use  of  full-motion  video,  animation,  and  audio 
on  networks.  It  is  possible  to  have  independent  usage  of  a  training  program  by  as  many  as  40 
users  on  a  single  local-area  network  (LAN)  with  complete  multimedia  functionality.  This  means 
that  entire  courses  can  be  made  available  anywhere  on  the  earth  with  appropriate  computer 
hardware,  including  the  ability  to  monitor  student  progress  remotely  on  a  remote  mainframe 
through  Internet  or  other  long-distance  network. 

Additionally,  the  tools  to  produce  highly  interactive  systems  have  improved  continuously,  re¬ 
sulting  in  more  and  better  system  and  learner  interactivity  in  the  training  programs  that  are  be¬ 
ing  developed.  Today,  there  is  really  no  question  of  the  instructional  efficacy  of  interactive 
technologies.  Regardless  of  subject  matter,  sophistication  of  audience,  or  complexity  of  train¬ 
ing  mission,  the  means  now  exist  to  create  a  computer-based  educational  solution  fully  com¬ 
petitive  with  any  traditional  classroom,  and  in  some  instances  competitive  with  the  ultimate 
instructional  system,  the  human  tutor. 

The  following  paragraph  is  an  abstract  taken  from  the  Institute  for  Defense  Analyses  (IDA) 
[Fletcher  90]: 

In  response  to  Congressional  direction,  a  quantitative,  analytical  review  was 
completed  of  interactive  videodisc  instruction  applied  in  Defense  training  and 
in  the  related  settings  of  industrial  training  and  higher  education.  Over  all 
instructional  settings  and  applications,  interactive  videodisc  instruction  was 
found  to  improve  achievement  by  about  0.50  standard  deviations  over  less 
interactive,  more  conventional  approaches  to  instruction.  This  improvement  is 
roughly  equivalent  to  increasing  the  achievement  of  students  at  the  50th 
percentile  to  that  of  students  currently  at  the  69th  percentile.  An  Improvement 
of  0.38  standard  deviation  was  observed  across  24  studies  in  military  training 
(roughly  an  increase  from  50th  to  65th  percentile  achievement).  An 
improvement  of  0.69  was  observed  across  14  studies  in  higher  education 
(roughly  an  increase  from  50th  to  75th  percentile  achievement).  Interactive 
videodisc  instruction  was  more  effective  the  more  the  interactive  features  of  the 
medium  were  used.  It  was  equally  effective  for  knowledge  and  performance 
outcomes.  It  was  less  costly  than  more  conventional  instruction.  Overall, 
interactive  videodisc  instruction  demonstrated  sufficient  utility  in  terms  of 
effectiveness,  cost,  and  acceptance  to  recommend  that  it  now  be  routineiy 
considered  and  used  in  Defense  training  and  education. 

In  general,  while  traditional  instruction  has  a  lower  cost  to  get  started,  the  cost  for  repeated 
deliveries  and  preparation  for  those  deliveries  will  be  relatively  constant.  By  contrast,  the  cost 
profile  for  developing  a  computer-based  solution  will  have  a  large  spike  in  the  initial  effort  that 
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continually  lowers  as  the  development  cost  is  amortized  over  the  lifetime  of  the  product  [ATP 


Taken  together — the  high  level  of  learner  interactivity,  the  richness  of  the  latest  multimedia 
features,  and  the  improvements  to  networks  and  CPUs— computer-based  training  solutions 
have  come  into  their  own.  It  is  now  possible  to  provide  students  with  an  engaging,  skill-based 
learning  experience  using  some  combination  of  multimedia  features  in  virtually  any  domain 
imaginable. 

3.1  Development  Issues  in  Multimedia 

There  are  at  least  three  well-known  models  in  the  interactive  multimedia  development  pro¬ 
cess: 

1 .  turnkey  in-house  development 

2.  turnkey  vendor  development 

3.  shared  in-house  and  vendor  development 

Option  number  3  is  the  most  common,  and  it  too  is  divided  by  degrees  that  range  from  partial 
to  total  participation  on  the  part  of  the  purchasing  organization.  In  any  of  these  cases,  the 
same  competencies  apply.  Sharing  of  responsibilities  between  organizations  increases  risk, 
from  communications,  differences  of  management  style,  physical  distances,  and  a  host  of  oth¬ 
er  factors.  In  almost  any  case,  the  decision  to  build  an  interactive  multimedia  system  will  be 
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an  organizationally  shared  responsibility.  It  is  therefore  in  the  interest  of  the  managing  organi¬ 
zation  to  understand  the  hardware  and  personnel  issues  associated  with  the  development 
process.  The  contracting  organization  that  is  sizing  up  potential  development  vendors  must 
understand  the  vendors’  capabilities  in  both  the  equipment  and  the  people  they  intend  to  em¬ 
ploy. 


3.2  Working  with  Vendors 

Because  developing  multimedia  is  inherently  complex,  the  contracting  organization  must  un¬ 
derstand  development  costs,  lead  times,  and  the  minimum  competencies  that  the  vendor 
should  be  able  to  demonstrate.  Sections  3.2.1  and  3.2.2  provide  a  set  of  issues  that  the  con¬ 
tracting  organization  should  look  for  in  selecting  a  vendor.  If  one  or  more  of  the  hardware  com¬ 
ponents  seem  suspect,  if  certain  team  roles  are  missing,  or  if  price  quotations  are  out  of  line 
with  what  is  shown  in  this  report,  the  vendor  is  probably  ill  suited  to  manage  an  interactive  mul¬ 
timedia  project  of  significant  size.  Naturally,  a  proven  track  record  is  persuasive,  but  experi¬ 
ence  alone  cannot  be  used  as  a  guarantee  of  success. 


3.2.1  The  Hardware 

The  hardware  for  creating  and  delivering  multimedia  applications  must  be  chosen  with  a  for¬ 
ward-thinking  perspective  so  that  the  solution  is  extensible  and  maintainable.  Even  with  care¬ 
ful  thinking,  market  changes  in  multimedia  systems  are  occurring  so  rapidly  that  even  the  most 
informed  developers  are  frequently  caught  by  surprise.  If  there  is  a  risk  to  pursuing  large-scale 
course  delivery  through  computer-based  instructional  strategies,  it  is  this  dynamism  of  the 
marketplace.  The  best  advice  in  pursuing  any  advanced  computer-based  training  system  is  to 
seek  out  the  expertise  of  an  experienced  consultant  and  vendor/developer  to  help  determine 
the  appropriate  configuration  for  the  organization’s  needs.  The  vendor  must  have  a  very 
strong  grasp  of  available  delivery  platforms  with  an  eye  to  coming  trends. 

3.2.1 .1  Delivery 

The  use  of  multimedia  narrows  the  field  to  two  primary  candidates  for  delivery:  Macintosh®- 
and  Intel-based  personal  computers  (PCs).  Currently  a  computer  capable  of  delivering  multi- 
media  programming  can  cost  as  little  as  $2,000  for  a  486-based  PC,  or  approximately  $4,000 
for  a  full  AA/  Macintosh  (16  mb  RAM,  500  mb  hard  drive,  CD-ROM).  The  prices  listed  would 
undoubtedly  be  lower  for  bulk  purchases.  Obviously  the  field  is  much  wider  for  systems  using 
only  text-based  training  or  text  with  limited  audio. 

3.2.1 .2  Development 

The  two  most  prominent  authoring  packages  on  the  market  are  Macromedia  Director®  (street 
price  approximately  $900)  and  Authorware®  (approximately  $7,000).  With  any  development 
platform,  the  possibility  of  creating  runtime  “players”  exists  for  cross-platform  delivery.  The  saf¬ 
er  bet  is  to  develop  the  software  on  a  platform  consistent  with  the  target  delivery  system. 

A  complete  professional  development  machine  for  multimedia  Intel-based  PC  costs  about 
$3,000.  This  would  typically  include  at  least  16  mb  of  RAM,  a  486-66  DX/2  processor,  a  1 ,024 
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X  768  monitor,  and  at  least  1  gigabyte  of  storage  memory  either  on  a  hard  drive  or  magneto¬ 
optical  drive.  An  equivalent  Macintosh-based  system  costs  approximately  $6000,  including 
PowerPC  processor,  a  high-quality  full-color  monitor,  and  a  gigabyte  of  memory.  Of  course  the 
printing  resources  in  either  case  add  anywhere  from  $1 ,500  to  $5,000,  depending  on  the  print¬ 
ing  requirements  of  the  project.  Combined  with  a  flatbed  and  other  image-scanning  devices, 
the  total  for  either  an  Intel-  or  Mac-based  multimedia  development  system  will  near  $10,000 
per  system.  In  general,  Intel-based  equipment  costs  less  than  Mac-based  equipment.  There 
is,  of  course,  ongoing  disagreement  among  developers  about  the  superiority  of  one  develop¬ 
ment  environment  over  the  other. 

In  the  normal  case,  there  is  at  least  one  machine  dedicated  to  a  project  of  small-to-medium 
size  (small  =  less  than  $50,000,  medium  =  $50,000-$300,000).  Projects  of  medium  size  will 
often  require  one  central  development  machine  and  anywhere  from  two  to  four  satellite  ma¬ 
chines  for  others  on  the  supporting  team:  writers,  image-preparation  personnel,  and  so  on. 
Therefore,  the  cost  of  hardware  for  any  medium-sized  development  project  will  most  likely  not 
be  less  than  $50,000  to  establish  a  very  basic  development  infrastructure.  Add  software  costs 
to  this,  and  the  initial  costs  for  a  basic  development  configuration  approaches  $70,000. 

3.2.2  People  Issues 

Any  multimedia  effort  is  a  team  venture.  The  developer  should  have  at  least  the  following  func¬ 
tional  roles:  project  manager,  content  expert,  writer,  computer  artist,  graphical  user-interface 
(GUI)  designer,  programmer,  and  instructional  designer.  The  contracting  organization  should 
view  weakness  in  any  of  these  areas  as  suspect.  Typically,  multimedia  vendors  are  small,  or 
are  small  subsets  of  larger  software  development  firms.  Established  methods  for  qualifying 
vendors  such  as  software  capability  evaluations  will  be  difficult  to  match  to  most  of  these  spe¬ 
cialty  companies,  which  are  known  in  the  multimedia  publishing  industry  as  “boutiques”  be¬ 
cause  of  their  small  size  and  resemblance  to  storefront  retail  operations. 

3.2.2.1  Development  Roles 

Project  manager:  Strong  project  management  skills  are  an  absolute  requirement  for  multime¬ 
dia  development  of  any  kind.  The  project  manager  must  be  adept  at  determining  the  scope  of 
the  project  before  scheduling  and  must  be  competent  at  developing  the  schedule  estimate 
based  on  the  scope.  The  project  manager  must  also  be  able  to  set  customer  expectations  ap¬ 
propriately. 

Content  expert:  The  content  expert  is  obviously  essential  to  the  creation  of  any  instructional 
material.  When  interactive  multimedia  technology  is  involved,  it  is  best  if  the  content  expert 
has  experience  with  and  understands  the  technology.  Having  a  content  expert  who  under¬ 
stands  the  technology  on  the  team  is  an  added  bonus  that  allows  communications  to  go  more 
quickly  and  smoothly.  The  content  expert  will  often  turn  up  insightful  methods  to  use  the  vari¬ 
ous  media  elements  to  best  advantage.  The  content  expert  can  often  quickly  discern  where 
the  media  is  interfering  with  the  message  being  conveyed. 
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Writer:  The  writer  often  serves  a  role  that  is  central  to  development  of  the  project,  often  step¬ 
ping  over  the  bounds  that  one  might  associate  with  writing,  at  first  glance.  The  writer  often  be¬ 
comes  involved  in  requirements  elicitation,  project  organization,  organization  of  materials, 
analysis  of  content,  and  other  roles. 

Computer  artist:  A  computer  artist  processes  or  creates  all  of  the  imagery  required  to  illustrate 
ideas,  backgrounds,  and  text  for  the  system.  If  the  organization  developing  the  computer- 
based  multimedia  system  is  competent  in  their  personnel  selection  process,  the  artist  will  also 
have  strong  ability  in  computer-interface  design.  Finding  these  skills  in  one  individual  can  be 
considered  very  lucky. 

GUI  designer:  This  individual  must  have  a  keen  intuitive  sense  of  design  as  well  as  formal 
training  on  how  to  understand  and  administer  field  trials  for  potential  system  users.  The  GUI 
designer  is  familiar  with  how  people  work  with  computers  and  how  to  use  the  computer  screen 
effectively. 

Programmer:  Ideally,  a  programmer  on  a  multimedia  development  team  are  capable  of  cross¬ 
ing  boundaries  with  art  and  design.  More  typical;  however,  is  the  case  in  which  these  functions 
are  separate,  requiring  team  discipline  and  consistency  in  maintaining  communications  be¬ 
tween  members.  There  is  often  the  need  for  someone  with  low-level  programming  skills  to  cre¬ 
ate  specialized  functionality  to  operate  under  higher-level  COTs  programs  (such  as 
Authorware  or  Macromedia  Director)  and  multimedia  elements  such  as  digital  video  or  large 
volumes  of  text. 

Instructional  designer:  Instructional  design  is  a  consultive  and  quality-control  function.  It  is  an 
invaluable  function  to  the  educational  design  of  any  multimedia  system.  The  instructional  de¬ 
signer  can  establish  and  evaluate  mid-project  user  tests  and  provide  useful  feedback  as  a  re¬ 
sult  of  those  evaluations.  An  instructional  designer  should  have  strong  facilitator  skills  and  the 
ability  to  elicit  requirements  for  the  system.  Often  the  instructional  designer  is  a  project  man¬ 
ager  as  well,  serving  a  central  position  in  specifying  the  system,  maintaining  communications 
between  project  team  members  and  the  system  customer. 

3.2.3  Development  Costing  Guidelines  for  Computer-based  Multimedia 

Based  on  estimates  from  Anderson  Consulting,  Inc.,  one  half-day  of  instruction  will  cost 
$277,500.  Anderson’s  breakdown  of  these  expenses  is 


Design  and  development 

(300  days  x  $500/day) 

$150,000 

Administration  and  management 

(60  days  x  $600/day) 

$36,000 

Video  subcontract 

$80,000 

Direct  reimbursables  (travel,  etc.) 

$11,500 
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Estimates  of  the  ratio  of  development  time  to  user  contact  hours  (also  called  seat  time)  are 
shown  as  follows: 

Low  400:1 

Medium  800:1 
High  1200:1 

Contrasted  against  a  small  multimedia-development  firm,  Visual  Symphony,  Inc.,"*  these  fig¬ 
ures  are  consistent.  Anderson  reports  their  typical  hourly  development  cost  at  $75  per  hour. 
Visual  Symphony  provides  a  range  from  $60  to  $100,  explained  by  the  variation  in  the  com¬ 
plexity  of  creating  user  interactions.  Visual  Symphony  adds  the  further  cost  of  front-end  anal¬ 
ysis  (scoping)  at  a  flat  rate  of  $1 0,000. 

3.3  System  Maintenance 

Because  of  the  rapidly  evolving  state  of  multimedia  today,  the  market  should  be  approached 
with  caution.  While  the  prices  of  delivery  platforms  are  becoming  less  expensive  and  the  soft¬ 
ware  to  run  on  those  platforms  is  becoming  more  available,  the  problems  of  relevance  still  re¬ 
main.  That  is,  off-the-shelf  multimedia  programs  are  unlikely  to  be  relevant  to  a  specific 
training  need  unless  an  instructor  can  adapt  the  commercial  software  package  to  fit  the  need. 
Development  of  multimedia  still  requires  specialized  development.  Allen  Communications,  a 
prominent  multimedia  developer,  has  produced  training  for  Air  Force  fighter  maintenance  and 
a  system  to  teach  fundamentals  of  running  a  business.  Anheuser-Busch  has  a  training  pro¬ 
gram  entitled  “Sales  Source  2000”  that  teaches  “beer-selling  basics.”  Electronic  Data  Sys¬ 
tems,  Inc.  has  a  program  that  teaches  inventory  policy,  procedures,  charging,  and 
troubleshooting.  Boeing  has  a  multimedia  program  teaching  maintenance  of  the  new  777  air¬ 
plane.  All  of  these  are  dynamic  programs  with  many  interesting  features.  Each  of  these  sys¬ 
tems  is  also  “tailor  made.”  So  far,  multimedia  development  is  a  job-shop  process  and  requires 
a  hand-in-hand  relationship  between  the  developer  and  the  developing  organization.  Further¬ 
more,  it  is  not  common  for  organizations  to  have  in-house  capabilities  to  develop  multimedia. 
There  are  exceptions  to  this,  however,  such  as  Anderson  Consulting. 

When  development  is  so  specialized,  maintenance  is  obviously  an  area  of  concern.  At  least 
three  appropriate  questions  to  ask  before  development  are 

1.  Who  bears  responsibility  to  maintain  and  repair  the  system  once  it  is  de¬ 
ployed? 

2.  Will  the  contracting  organization  commit  to  learning  to  maintain  the  system, 
or  will  the  developer  be  responsible? 

3.  How  volatile  is  the  subject  matter? 


A  small  multimedia  development  company  in  based  in  Pittsburgh,  Pa. 
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Each  of  these  questions  must  be  carefully  considered. 

3.4  System  Life  Cycle:  Long  Term  vs.  Short  Term 

Clearly  multimedia  is  an  appropriate  choice  for  a  course  with  a  long-term  need.  In  the  best 
case,  a  careful  front-end  analysis  will  reveal  the  longevity  of  the  need.  The  contracting  orga¬ 
nization  should  establish  a  satisfactory  relationship  with  a  trusted  multimedia  developer.  Such 
a  partnership  should  enable  courses  to  be  created  with  maximum  reuse  of  existing  materials 
and  code  modules  developed  in  initial  projects. 

In  any  case,  the  contracting  organization  should  structure  its  agreement  with  the  developer  to 
retain  rights  to  any  software  created  during  the  project.  Such  an  arrangement  is  typically  doc¬ 
umented  in  a  standard  agreement  that  permits  the  developer  to  reuse  code  on  future  projects 
for  other  clients  but  allows  the  contracting  organization  to  have  full  access  to  any  part  of  sys¬ 
tem  code  at  all  times. 


3.5  Related  Solutions  in  the  Multimedia  Famiiy 

The  following  descriptions  involve  one  or  more  media  elements  that  can  be  categorized  as 
multimedia.  Any  one  of  these  systems  can  present  digital  sound,  images,  text,  video,  or  music, 
either  in  a  piecemeal  fashion  or  as  an  integrated  instructional  system. 

3.5.1  Electronic  Performance  Support 

EPSS  are  a  kind  of  computer-based  multimedia  that  are  integrated  directly  into  the  context  of 
work  being  performed.  A  recent  application  is  a  spreadsheet  created  by  Lotus®.  If,  while  using 
Lotus  1-2-3®,  the  user  has  a  question  on  how  to  create  a  macro,  a  training  module  can  be 
invoked.  Because  it  offers  so  many  possibilities  to  optimize  on  learning  and  performance  out¬ 
comes,  electronic  performance  support  is  gaining  a  solid  following  among  technologically  as¬ 
tute  educators.  Again,  the  same  “platform”  learning  factors  are  in  play:  interactivity,  situated 
experience,  and  learner  adaptivity. 

The  issues  surrounding  EPSS  are  not  intrinsically  different  from  multimedia  generally.  Al¬ 
though  these  systems  offer  a  great  educational  payoff,  there  are  not  many  of  them  in  exist¬ 
ence,  and  creating  them  requires  the  proper  front-end  analysis  to  justify  an  organizational 
commitment  to  their  development.  Otherwise,  there  are  exceptional  examples  for  specific  tar¬ 
get  domains  such  as  the  Lotus  example  above. 

3.5.2  Hypermedia  Information  Services 

Hypermedia  is  computer  jargon  describing  the  interlinkage  of  text  and  symbols  to  provide  sys¬ 
tem  users  with  the  ability  to  move  from  one  link  to  the  next.  This  linking  of  information  is  in¬ 
tended  to  provide  informational  points  of  departure  through  large  bodies  of  information. 

The  Worldwide  Web  (WWW)  is  now  a  prime  example  of  hypermedia,  whose  use  is  expanding 
daily.  With  Internet  hosts  increasing  at  a  rate  of  approximately  7  percent  per  month,  network 
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usage  is  rapidly  becoming  commonplace  and  will  undoubtedly  be  as  common  as  the  tele¬ 
phone  in  the  very  near  future.  Corresponding  to  this  growth  is  the  number  of  available  infor¬ 
mation  sources  on  the  network.  Currently  there  are  WWW  sites  for  information  on  the  National 
Aeronautics  and  Space  Administration  (NASA),  American  Telephone  and  Telegraph  (AT&T), 
Apple  Computer,  Software  Engineering  Institute,  major  U.S.  magazines,  the  U.S.  Patent  Of¬ 
fice,  and  many  others.  Creating  curricula  to  use  these  resources  is  a  new  idea.  It  is  possible 
to  download  entire  multimedia  presentations  from  the  network.  To  date  there  is  not  a  great 
deal  known  about  teaching  courses  using  the  Internet  interactively. 

Currently,  courses  taught  using  the  Internet  are  new  and  varied.  One  course  developed  at  Gal- 
laudet  University  has  linked  deaf  students  with  another  class  at  Rochester  Institute  of  Tech¬ 
nology;  however,  interactive  use  of  the  Internet  for  instruction  is  experimental.  Its  use  as  a 
resource  with  network  news  services,  email,  and  specially  created  newsgroups  is  more  than 
ten  years  old  however,  and  in  conjunction  with  rapidly  developing  WWW  sources,  the  Internet 
stands  to  be  an  ever-richer  resource  for  any  instructor  who  has  ready  network  access. 

3.5.3  Just-In-Time  Lecture  (JIT) 

JIT  lecture  is  a  very  straightforward  use  of  various  multimedia  technologies  to  store  and  re¬ 
trieve  lectures  for  access.  The  Carnegie  Mellon  University  (CMU)  Multimedia  Lab  is  develop¬ 
ing  a  standard  process  by  which  lecturers  can  easily  record  their  own  lectures  in  digital  video 
to  be  organized  by  a  topical  outline  for  random  access  to  lecture  topics  by  students.  The  sys¬ 
tem  is  integrated  into  the  CMU  campuswide  network,  and  students  are  able  to  submit  ques¬ 
tions  to  the  instructor  (or  teaching  assistant)  through  email.  If  a  question  is  submitted  by 
several  students,  the  instructor  will  add  more  material  to  the  lecture;  otherwise  the  question  is 
answered  individually.  Currently  the  system  permits  instructors  to  provide  visual  and  audible 
email  responses.  The  objective  of  the  JIT  development  effort  at  this  time  is  to  reduce  the  cur¬ 
rent  lecture  production  time  from  40  hours  to  20  within  the  next  year. 

3.5.4  Intelligent  Tutoring 

Intelligent  Tutoring  Systems  (ITSs)  are  computer-based  instructional  systems  that  diagnose 
the  responses  of  a  user  in  order  to  make  incisive  remediations  as  a  human  tutor  would.  Al¬ 
though  all  indications  are  that  multimedia  would  do  nothing  but  improve  such  systems,  ITSs 
can  be  used  with  or  without  multimedia.  In  contrast  to  standard  computer-based  training,  ITSs 
do  not  follow  a  fixed  set  of  responses  to  student  errors.  Their  “intelligence”  is  in  figuring  out 
the  “cognitive  malady”  of  the  student  in  the  context  of  instruction.  ITSs  attempt  to  deduce  why 
the  student  is  making  a  specific  error,  for  example  a  math  tutoring  system,  would  try  to  deter¬ 
mine  why  a  student  was  failing  to  add  two  numbers  correctly.  It  would  use  analysis  rules  to 
make  instructional  decisions  related  to  the  domain.  For  example,the  student  might  not  per¬ 
forming  a  “carry”  properly,  or  perhaps  is  not  understanding  “place  value.”  In  other  words,  the 
system  reasons  as  a  human  tutor  would  about  the  causes  of  a  student’s  problems  and  at¬ 
tempts  to  match  this  “diagnosis”  to  insightful  tips  and  remedial  commentary. 
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Building  intelligence  into  a  computer-based  instructional  system  requires  a  skilled  program¬ 
mer  who  is  familiar  with  expert-system  and  artificiai-inteliigence-based  programming.  Devel¬ 
opment  costs  have  decreased  as  new  tools  have  been  developed,  and  the  continual 
improvement  of  delivery  systems  is  creating  new  possibilities  for  developing  this  type  of  sys¬ 
tem  more  easily. 

3.5.5  Text-Based  Computer-Based  Training  (CBT) 

Most  online  tutorials  (for  example  those  using  Intel-based  PCs)  are  text  based.  While  this  kind 
of  training  is  far  less  demanding  in  development  than  multimedia,  its  effectiveness  as  a  train¬ 
ing  vehicle  is  limited  by  its  lack  of  the  richer  features  of  full  multimedia  capability.  Lack  of  full- 
motion  video  and  images,  the  value  of  which  has  been  described  by  Christel  [Christel  94], 
places  definite  limitations  on  the  educational  environment.  While  the  effectiveness  of  interac¬ 
tive  video  systems  has  been  proven,  I  know  of  no  major  studies  pertaining  to  the  effectiveness 
of  text-based,  online  tutorials.  To  the  extent  that  they  are  interactive,  they  will  undoubtedly 
have  instructional  value;  however  this  would  in  most  cases  be  highly  procedural  training. 


3.6  The  Multimedia  Family  of  Solutions  and  Travel  Reduction 

Because  of  its  potential  distribution  to  any  properly  configured  desktop  computer  system,  in¬ 
teractive  multimedia  can  virtually  eliminate  travel  to  classrooms.  Multimedia  can  be  delivered 
with  or  without  a  connection  to  a  network.  When  delivery  to  a  network  is  not  feasible,  a  com¬ 
plete  multimedia  training  program  can  be  delivered  on  CD-ROM. 

Figure  2  shows  a  currently  available  network  configuration  that  permits  full  audio-visual  func¬ 
tionality,  including  full-motion  video  on  a  40-user  LAN.  This  is  not  hypothetical.  Starlight  Net¬ 
works®  produces  this  hardware,  and  there  is  a  slightly  less  powerful  and  reliable  version 
produced  for  Novell®.  In  this  configuration,  students  can  work  at  their  individual  workstations 
independently  in  a  cluster  configuration  or  at  widely  separated  sites. 

Real-World  Example 

When  a  network  is  used,  results  of  training  can  be  downloaded  to  a  mainframe  using  a  mo¬ 
dem.  This  type  of  configuration  is  currently  being  used  by  CSX  Transportation  for  conductor 
training.  Their  system  permits  conductors  to  take  portions  of  the  required  8-hour  program  as 
they  travel  over  a  span  of  26  cities.  The  CSX  system  includes  full-motion  video,  animations, 
audio,  and  interactive  examples  and  testing  to  teach  the  use  of  a  new  computer  technology 
for  work-order  report  that  is  to  be  installed  in  all  CSX  train  cabs. 
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The  network  configuration  used  for  the  CSX  application  is  not  particularly  complex.  The  fol¬ 
lowing  reflects  current  prices  for  one  40-user  multimedia  workstation  configuration: 

•  $2,500  per  workstation 

•  $8000  per  64  mb  RAM  server 

•  $4000  per  6  gigabyte  hard  drive 

•  $25,000  network  driver  software 

•  $8000  per  switched  hub 

When  the  system  requires  five  or  fewer  users  per  LAN  (as  with  CSX),  the  switched  hub  is  not 
needed. 
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Multimedia  Server  Configuration 


Student  Workstation 


Figure  2:  Multimedia  Network  Configuration 
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As  a  distance-learning  solution,  the  CSX  example  demonstrates  that  training  that  utilizes 
state-of-the-art  interactive  multimedia  can  be  delivered  to  sites  that  are  separated  by  hun¬ 
dreds  of  miles,  while  at  the  same  time  providing  centralized  performance  monitoring.  Virtually 
any  interactive  multimedia  that  can  be  used  in  a  stand-alone  system  can  now  also  provide  the 
benefits  of  the  network  as  well. 

3.7  Other  Potential  Distance-Learning  Solutions 

Outside  the  realm  of  the  interactive  computer-based  systems,  there  are  still  the  “tried-and- 
true”  methods  of  course  delivery.  Videotape,  satellite,  and  travelling  lecturers  have  well-known 
positive  features  as  well  as  certain  clear  limitations.  For  better  or  worse,  these  methods  rep¬ 
resent  a  “stepping  back”  from  the  revolutionary  change  implied  in  the  current  wave  of  advanc¬ 
es  in  the  computer-based  systems.  I  consider  these  methods  in  the  following  paragraphs. 

3.7.1  Videotape 

Videotape  has  been  available  as  an  instructional  technology  for  many  years  and  provides  us 
with  many  useful  lessons  in  understanding  the  educational  value  of  video  in  the  new  comput¬ 
er-based  technologies.  For  instance,  interactivity  is  essential.  An  inherent  weakness  of  video¬ 
tape  is  its  slow  rewind  and  fast-forward  capability.  While  video  can  show  us  many  things,  it  is 
only  as  good  as  the  methods  used  to  present  the  content.  This  holds  true  for  the  fanciest  of 
technological  solutions,  from  multimedia  to  intelligent  tutoring  to  satellite  instruction — success 
depends  on  the  quality  of  the  production.  While  this  may  seem  an  obvious  point,  it  is  remark¬ 
able  how  often  this  precept  is  ignored.  The  “talking  head”  video  is  one  of  the  most  common  in 
educational  fare  and  provides  us  with  a  landmark  of  “things  to  be  avoided”  as  we  venture  down 
the  technological  highway. 

3.7.2  Satellite 

Satellite  is  a  well-understood  educational  technology.  Although  it  is  not  often  taken  advantage 
of,  the  potential  for  interactivity  is  great  for  satellite  courses.  For  example,  the  satellite  program 
managed  through  the  Air  Force  Institute  of  Technology  (ART)  offers  brief  instruction  on  the 
development  of  course  materials  for  satellite  broadcast.  Although  the  visual  feed  is  one  way, 
AFIT  provides  conference  audio  through  phone  lines.  This  enables  the  instructor  to  commu¬ 
nicate  with  students  and  enables  students  to  hear  answers  and  explanations  given  to  one  an¬ 
other.  At  a  cost  of  $200  per  hour,  not  including  a  $50  technician's  fee  and  course  development 
costs,  satellite  is  now  facing  stiff  competition  from  computer-based  distance-learning  solutions 
in  terms  of  the  educational  value  added. 

It  is  possible  to  have  full,  two-way  communications  through  satellite  at  a  correspondingly  high¬ 
er  price  than  one-way  video.  Obviously,  the  interactive  capabilities  are  improved  with  both  two- 
way  audio  and  two-way  video;  however,  unless  the  audio  and  video  are  operating  at  a  high 
display  rate,  communications  can  seem  confusing  and  garbled.  Real-time  satellite  communi¬ 
cation  is  only  as  good  as  the  preparation  of  the  materials  designed  for  the  course  and  the  pre- 
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sentation  skills  of  the  instructor.  With  no  technical  problems,  two-way  satellite  provides  the 
possibility  of  a  “virtual  classroom.” 

3.7.3  Lecturers  On  Site 

Rather  than  taking  the  student  to  the  course,  we  can  also  send  the  course  to  the  student.  As 
in  one-way  satellite  delivery,  the  lecturer’s  use  of  interactivity  can  vary  widely.  As  a  facilitator, 
a  classroom  instructor  can  provide  the  highest  levels  of  student  engagement  as  are  possible 
beyond  one-to-one  tutoring.  The  risk,  however,  is  that  the  lecturer  will  use  little  or  no  interac¬ 
tivity.  Many  instructors  are  simply  not  comfortable  with  high  levels  of  interaction  with  students. 
Classroom  lectures  are  only  as  good  as  the  planning  and  design  that  goes  into  them,  and  even 
with  excellent  designs,  a  lackluster  instructor  can  ruin  the  experience.  In  general,  any  measur¬ 
able  educational  outcome  can  be  achieved  using  computer-based  technology.  Further,  com¬ 
puter-based  technologies  guarantee  a  stable  style  and  manner  of  instruction  over  time  and  for 
any  number  of  classrooms.  Additionally,  articles  in  major  interactive  multimedia  journals  are 
providing  examples  of  this  technology  used  as  a  classroom  support.® 


A  recent  example  appears  in  Multimedia  Todays,  1  (Jan.-March  1995):  57*60. 
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4  A  Technology-Specific  Analysis 


Figure  3  matches  each  of  1 0  educational  delivery  methods  with  factors  related  to  instructional 
effectiveness,  development  complexity,  and  associated  costs.  This  figure  is  an  educated 
guess  of  the  relationships  known  and  represents  a  combination  of  research,  experience,  and 
rules  of  thumb  in  the  absence  of  hard  science.  Many  of  these  technologies  are  new  and  de¬ 
veloping  rapidly.  Most  of  the  data  supporting  this  figure  comes  from  cases  and  field  reports 
cited  throughout  this  report.  The  purpose  of  the  figure  is  to  provide  decision  makers  with  a 
grounded  overview  of  technologies  and  relevant  implementation  factors  as  of  today.  Because 
these  are  fast-emerging  technologies,  the  data  will  likely  be  quickly  dated;  nevertheless,  the 
data  can  provide  valuable  insight  for  today’s  decision  space. 

The  following  descriptions  correspond  to  the  factors  listed  across  the  top  of  Figure  3. 

4.1  Potential  Level  of  Interactivity 

This  category  defines  interactivity  as  the  potential  amount  of  time  the  learner  is  engaged  in  a 
task,  actively  pursuing  an  instructional  goal.  As  stated  earlier,  educational  researchers  refer 
to  interactivity  in  various  ways,  such  as  “time  on  task.”  Informally,  we  can  think  of  various  pos¬ 
itive  learning  experiences  in  which  students  were  alert  and  contributing  to  the  educational  pro¬ 
cess.  The  question  this  analysis  seeks  to  answer  is  “To  what  extent  is  the  technology  capable 
of  maintaining  an  interactive  experience  with  the  student?” 

Of  the  systems  listed,  seven  are  fully  capable  of  engaging  students  in  a  rich  instructional  dia¬ 
logue.  Videotape  is  obviously  a  one-way  communication:  while  students  may  be  engaged  by 
a  well-produced  tape,  for  the  purposes  of  this  report  I  would  not  consider  videotape  interactive 
in  the  same  sense  as  a  dynamic  classroom  or  multimedia  experience  could  be. 


4.2  Project  Management  in  Development 

There  are  differences  in  the  amount  of  time  required  to  manage  the  development  of  any  new 
product.  Instructional  technologies  are  no  exception,  and  today  the  differences  are  so  wildly 
divergent  that  it  is  very  easy  to  be  caught  unprepared.  I  intend  this  category  primarily  as  a  gen¬ 
eral  warning  about  the  amount  of  time  needed  to  properly  develop  computer-based  instruc¬ 
tional  systems.  With  or  without  the  use  of  technologies,  course  development  is  time 
consuming  and  often  difficult.  Technologies  of  any  kind  add  to  this  difficulty  and  complexity.  In 
particular,  interactive  multimedia  requires  intensive  planning  and  management  activity,  pref¬ 
erably  with  the  help  of  software  tools. 

4.3  Availability  of  Delivery  Platforms 

This  category  defines  the  available  means  by  which  the  instruction  is  delivered  to  the  student. 
Specifically,  this  category  largely  refers  to  hardware  availability:  computers,  satellites,  VCRs, 
overhead  projectors,  satellite  downlinks,  and  so  on.  Remembering  that  Figure  3  is  a  compar- 
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ison  chart,  we  can  say  that  lecturers  are  more  readily  available  to  deliver  instruction  than  com¬ 
puter-based  multimedia  instructional  systems,  for  example  (even  though  this  may  not  be  the 
case  in  the  near  future).  It  is  also  important  to  note  that  the  military  has  a  large  satellite  infra¬ 
structure  available  for  nationwide  course  delivery,  which  is  more  readily  available  for  course 
delivery  than  computer-based  instructional-delivery  platforms. 

4.4  Maintenance  Estimate 

Maintenance  is  defined  as  the  required  modifications  made  to  existing  course  materials.  The 
circles  in  this  category  provide  a  comparison  of  the  amount  of  this  activity  required  for  the 
types  of  technologies  listed.  They  do  not  answer  the  question  of  how  many  modifications  will 
be  required  for  any  given  course;  rather,  they  are  meant  to  make  it  possible  to  compare  how 
much  modification  effort  will  be  required  once  the  course  exists  as  a  product.  For  example, 
even  though  individuals  often  share  their  lecture  and  course  materials,  these  often  undergo 
significant  changes  from  one  instructor  to  the  next  depending  on  instructors’  preferences  in 
emphasis  and  so  forth.  In  general,  computer-based  technologies  are  less  modifiable  than  hu¬ 
man-dependent  instructional  systems  (including  satellite).  This  implies  that 

•  Because  of  the  relative  difficulty  in  modifying  computer-based  instruction, 
course  developers  should  carefully  prepare  materials  for  long-term  use. 

•  While  lecturers  have  the  advantage  of  wide  flexibility  in  modifying 
instructional  materials,  modification  adds  the  risk  that  key  instructional  issues 
will  be  inconsistently  addressed  over  the  lifetime  of  a  course. 

In  light  of  these  considerations,  for  better  or  worse,  there  will  in  general  be  less  maintenance 
of  multimedia-based  instructional  systems  and  more  maintenance  for  other  human  and  com¬ 
puter-based  instructional  systems.  Of  course,  content  volatility  is  a  consideration  with  respect 
to  any  course  development  effort;  however,  courses  are  typically  a  mix  of  volatile  elements 
and  static  principles.  Clearly,  where  substantial  amounts  of  the  content  are  changeable  on  a 
monthly  basis,  any  computer-based  technology  will  become  a  questionable  option. 


4.5  Delivery  Cost 

The  cost  of  delivering  a  course  includes  personnel  required  in  the  classroom  at  the  time  of  de¬ 
livery,  the  cost  of  any  printing  or  creation  of  materials  required  for  the  delivery,  and  any  other 
overhead  specific  to  the  classroom.  In  Figure  3,  all  computer-based  delivery  methods  are  rat¬ 
ed  as  relatively  low.  The  on-site  lecturer  is  higher  because  of  required  materials  preparation 
and  the  lecturer’s  time.  Satellite  delivery  is  ranked  highest  because  of  the  cost  per  hour  for  use 
of  the  facilities  in  addition  to  those  costs  associated  with  the  lecturer  and  materials.  Videotape 
is  ranked  lowest  because  it  can  be  shipped  to  any  site  with  access  to  a  VCR  and  observed 
without  cost  other  than  the  time  taken  to  view  the  tape. 
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4.6  Overall  Development  Cost 

This  category  refers  to  expenses  related  to  the  design,  development,  and  production  of  edu¬ 
cation  and  training  relative  to  delivery  methods.  Currently,  there  is  a  much  development  asso¬ 
ciated  with  any  computer-based  instructional  education  strategy.  It  is  therefore  an  important 
consideration  in  planning  for  this  kind  of  technology.  Among  the  computer-based  delivery 
methods,  any  that  use  multimedia  will  be  at  the  highest  end  of  development  cost  because  of 
the  many  detailed  activities  involved.  Comparing  two  computer-based  methods,  text-based 
CBT  and  multimedia,  both  will  be  complicated  to  produce;  however,  because  text-based  CBT 
involves  only  the  use  of  ascii  text,  it  will  be  far  less  difficult  to  create  than  multimedia.  Using 
multimedia  implies  the  creation  or  selection  of  photographs,  voice,  music,  or  video.  For  each 
element,  there  are  processing  steps  involved  and  decisions,  such  as  the  scanning  of  photos, 
shooting  of  video  and  so  on. 

4.7  Potential  Reduction  of  Travel 

Each  of  the  delivery  methods  listed  has  some  potential  to  reduce  the  travel  time.  Regardless 
of  the  method  of  course  delivery  selected,  we  can  assume  that  course  developers  hope  to 
maintain  high-quality  instructional  outcomes.  In  light  of  this  assumption,  the  shading  of  the  cir¬ 
cles  in  this  category  answer  the  question  “Which  technology  is  best  able  to  retain  the  highest 
level  of  instructional  outcome  even  when  the  students  cannot  travel  to  the  lecture  itself?”  Irre¬ 
spective  of  distances,  computers  will  deliver  the  same  level  of  instructional  quality  that  was 
programmed  into  them  in  the  first  place.  The  same  can  be  said  for  satellite  systems:  depend¬ 
ing  on  the  instructional  effectiveness  of  the  instructor  using  a  satellite,  distance  will  not  change 
the  outcome.® 

4.8  Potential  for  Materials  Reuse 

Reuse  of  materials  is  of  obvious  importance  in  product  development.  I  found  no  evidence  to 
suggest  that  one  delivery  method  provides  greater  materials  reusability  than  another.  Com¬ 
puter-based  instructional  systems  allow  for  the  reUse  of  certain  code  routines  used  for  student 
evaluation.  Computer-based  multimedia  can  provide  the  opportunity  for  reusing  image  librar¬ 
ies,  evaluation  routines,  video,  audio,  and  sound,  depending  on  the  nature  of  the  course  and 
how  generic  the  need.  More  human-dependent  instructional  methods,  such  as  satellite,  pro¬ 
vide  for  the  possibility  of  reusing  lecture  materials  and  even  interaction  methods  depending  on 
the  inclination  of  the  instructors  to  do  so. 


4.9  Overall  Estimate  of  Instructional  Effectiveness 

Based  on  the  instructional  factors  provided  earlier  in  the  report,  my  intention  for  this  category 
is  to  provide  a  general  view  of  the  capability  of  the  delivery  methods  to  be  instructionally  ef¬ 
fective.  My  assumption  here  is  that  the  more  the  instructional  medium  is  capable  of  providing 


®  In  the  absence  of  scientific  data  to  refute  the  idea,  this  seems  like  a  reasonable  assumption. 
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sustained  involvement  and  attention  to  specified  learning  tasks,  the  more  effective  it  is.  For 
example,  videotape,  even  when  used  with  a  specialized  interactive  videotape  machine,  can¬ 
not  provide  for  the  same  degree  of  student  involvement  because  of  the  relatively  long  seek 
times  associated  with  it.^  The  student  has  to  endure  lengthy  waits  while  the  machine  rewinds 
the  tape.  While  two-way  satellite  would  seem  to  be  able  to  provide  near-classroom-style  fidel¬ 
ity,  in  fact  it  does  not,  because  of  variations  in  transmission  speeds.  These  variations  are  often 
disruptive  and  distracting  to  dialogue.  Intelligent  tutors,  EPSS,  multimedia,  and  human-based 
on-site  instruction  all  have  the  potential  for  rich  and  attention-demanding  interaction  with  the 
learner. 

4.10  Potential  Learner  Adaptability 

The  human  tutor  is  an  accepted  baseline  measure  of  instructional  effectiveness  against  which 
instructional  technology  has  often  been  compared  [Anderson  85].  Tutors  are  able  to  analyze 
the  difficulties  of  their  students  and  make  adaptations  to  their  teaching  strategies  as  circum¬ 
stances  arise.  The  rankings  listed  show  that  four  of  the  computer-based  technologies  are  the 
most  highly  adaptable  to  the  individual  learner.  Because  an  instructor’s  ability  to  adapt  to  the 
individual  is  constrained  by  the  number  of  students  enrolled,  on-site  lecturers  have  less  adapt¬ 
ability.  Of  course  this  also  applies  to  satellite  technology  because  of  its  relative  dependence 
on  the  instructor  and  class  size  in  addition  to  the  separation  of  instructor  from  the  class.  Text- 
based  CBT  has  the  potential  for  adaptability,  but  adding  this  capability  would  push  this  tech¬ 
nology  into  the  intelligent  tutoring  category.  (To  reiterate,  text-based  CBT  is  defined  for  the 
purposes  of  this  report  as  a  label  to  describe  text-based  computer  tutorials  with  branching  ca¬ 
pability,  having  no  capacity  to  adapt  to  specific  learners.)  Just-in-time  lecture  provides  random 
access  to  segments  of  lectures,  offering  some  learner  adaptability;  however,  these  adapta¬ 
tions  depend  on  the  learner’s  ability  to  seek  out  the  information.  By  contrast,  videotape  simply 
does  not  adapt. 


Despite  certain  specific  instances  where  individuals  have  created  usable  random-access  videotape  systems, 
this  technology  can  rightly  be  thought  of  as  obsolete. 
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Instructional  Systems  and  Implementation  Factors 


Figure  3:  Estimated  Instructional  and  Development  Factors 
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5  Summary,  Recommendations,  and  Conclusions 


Based  on  the  discussion  of  core  educational  issues  and  the  available  technologies,  our  task 
becomes  one  of  “mix  and  match,”  based  on  how  much  money  there  is  to  spend  against  the 
learning  outcomes  that  are  to  be  achieved. 

5.1  Summary 

There  are  two  primary  considerations  in  selecting  an  instructional  technology  solution:  learn¬ 
ing  factors  and  delivery  logistics.  The  technology  selection  objective,  therefore,  is  to  optimize 
on  the  best  and  most  appropriate  learning  factors  while  minimizing  the  negative  effects  of  the 
logistics  involved  in  delivering  instruction. 

Prominent  among  the  learning  factors  are 

•  interactivity 

•  learner  adaptability 

•  situated  learning 

•  retention 

•  volatility  of  instructional  content 

•  consistency  of  training  quality 

•  overall  development  cost 

•  existing  hardware  and  software  infrastructure 

•  ease  of  maintenance 

•  potential  for  course  participant  travel  reduction 

•  potential  for  course  content  reuse 

To  provide  interactive  multimedia  functionality,  the  computer  infrastructure  must  be  adequate. 
Depending  on  the  types  of  multimedia  services  that  an  organization  wishes  to  provide,  the 
possible  delivery  mechanisms  can  range  from  $1 ,700  to  $6,000  apiece.  It  is  possible  to  pro¬ 
vide  an  infrastructure  capable  of  supporting  all  of  the  top-of-the-line  multimedia  features,  such 
as  full-motion  (hardware-assisted)  video,  in  a  LAN  configuration,  or  to  provide  only  a  limited 
set  of  these  features,  where  a  LAN  may  or  may  not  be  a  part  of  the  system  configuration.  A 
hypothetical  10-user  LAN  supporting  full-motion  hardware-assisted  video  (30  frames  per  sec¬ 
ond)  as  well  as  any  other  available  multimedia  feature  will  cost  approximately  $65,000.  This 
price  would  include  10  workstations,  a  6-gigabyte  server  and  set-up  charges. 

Using  Intel-based  systems,  full-multimedia  capability  can  be  made  available  including  an  in¬ 
board  CD-ROM  drive  and  modem  on  any  desktop  for  under  $2000.  A  machine  having  these 
capabilities  permits  access  to  existing  WWW  offerings,  the  Internet,  and  the  use  of  CD-ROMs. 
This  is  not  to  paint  an  overly  rosy  picture.  Obtaining  access  to  the  Internet  and  assuring  rea¬ 
sonable  system  performance  (speed)  in  downloading  images  and  other  types  of  large-data 
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items  will  require  the  purchase  of  the  appropriate  high-bandwidth  connections,®  adequate  lo¬ 
cal  host  storage,®  server  software,  and  any  other  ancillary  support  devices  such  as  Ethernet 
cards  for  user  machines. 

The  network  technologies  required  to  deliver  WWW  and  corresponding  multimedia  function¬ 
ality  to  the  classroom  are  well  understood  and  will  require  a  separate  analysis  considering  the 
numbers  of  students  that  would  participate  in  pilot  courses.  The  major  technical  consider¬ 
ations  in  this  regard  will  revolve  around  the  number  of  users  and  their  influence  on  network 
performance.  Solutions  must  be  scalable  to  larger  numbers  of  students,  with  an  eye  to  hard¬ 
ware  improvements  and  upgrades. 

5.2  High-Level  Recommendations 

5.2.1  Build  an  Educational  Infrastructure 

The  purpose  of  this  infrastructure  is  to  support  WWW  access  and  provide  for  potential  multi- 
media  course  distribution.  Given  the  rapidly  growing  availability  and  shrinking  cost  of  comput¬ 
er-based  multimedia  hardware  and  software,  the  time  is  ripe  to  establish  a  technological 
infrastructure  that  will  open  the  door  to  the  full  range  of  multimedia  offerings  and  anything  on 
the  WWW — all  at  one  time.  Among  the  rapidly  growing  number  of  subscribers  to  the  WWW, 
there  is  no  argument  as  to  the  horizons  that  this  technology  is  opening,  especially  from  an  ed¬ 
ucational  standpoint.  Further,  there  is  already  a  well-established  precedent  among  American 
universities,  where  the  value  of  the  networks  is  inarguable. 

5.2.2  Assess  Information  Repositories  to  Be  Made  Available  Through  the 
WWW 

There  also  must  be  a  forward-thinking  assessment  of  the  target  workstation  suitable  for  use 
both  online  and  as  a  stand-alone  system  with  full  multimedia  capability.  It  is  now  possible  to 
purchase  a  complete  multimedia  workstation  with  large  storage  capacity  and  inboard  modem 
for  $1 ,700  in  the  consumer  marketplace.  This  cost  implies  that  with  an  understanding  of  the 
geographical  distribution  of  the  student  population,  that  both  LAN-based  and  stand-alone  sys¬ 
tem  configurations  are  feasible. 

5.2.3  Pick  a  First  Target  for  Computer-Based  Multimedia 

Establish  a  candidate  project  for  implementation  as  a  multimedia  course.  Remembering  that 
a  rule  of  thumb  is  that  complexity  costs  more,  determine  a  relatively  long  life-cycle,  low-com¬ 
plexity  course  for  implementation. 
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greater  than  2  gigabytes  at  a  minimum 
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5.3  Lower  Level  Suggestions 

The  following  represent  four  possible  approaches,  as  well  as  a  framework  for  thinking  about 
strategies  of  implementation.  These  are  not  exhaustive,  but  each  represents  a  plausible  set 
of  choices,  assuming  that  the  basic  system  infrastructure  is  in  place.  As  stated  in  Section  1 .2, 
assumptions  about  the  organizations  to  which  these  suggestions  apply  are  the  following: 

•  A  certain  amount  of  instructional  infrastructure  is  already  in  place. 

-  Satellite  hardware  and  system  support  is  readily  available  for  wide-scale 
transmission  of  one-way  and  potentially  two-way  educational  offerings. 

-  The  existing  computer  hardware  base  is  composed  largely  of  Intel® 
microprocessor-based  machines,  capable  of  delivering  text-based 
instruction  [computer-based  training  (CBT)]. 

•  The  organization  is  willing  to  explore  the  possibility  of  expanding  the  existing 
infrastructure  to  include  technologies  capable  of  interactive  multimedia'* °  and 
other  advanced  computer-based  instructional  solutions. 

5.3.1  Strategy  1 :  Network-Distributed  Computer-Based  Training 

This  strategy  consists  of  downloading  low-level  interactive  training  courses  to  LANs  for  use  at 
student  terminals. 

Objective 

Engage  in  a  slow  migration  to  computer-based  course  delivery.  This  plan  would  optimize  ex¬ 
isting  capacity  while  building  for  greater  infrastructure  capacity  in  a  “slow-growth”  scenario. 

Features 

Many  sites  across  the  U.  S.  currently  have  386-level,  Windows-compatible  computers  avail¬ 
able  for  course  distribution.  These  machines  are  capable  of  communication  with  the  Internet 
or  LAN. 

Advantages 

Given  the  current  state  of  practice,  this  option  could  be  put  in  place  quickly.  It  would  enable  an 
organization  to  run  test  cases  to  determine  the  efficacy  of  the  approach  on  a  limited  basis. 

While  the  development  cost  for  any  interactive,  computer-based  instructional  solution  always 
has  the  drawback  of  initial  development  cost,  text-based  CBT  is  the  least  expensive  of  the  ex¬ 
isting  computer-based  instructional  approaches  in  this  regard.  And  it  holds  the  edge  of  inter¬ 
activity  as  an  instructional  approach. 

Limitations 

Text-based  CBT  is  among  the  least  interesting  of  current  possibilities  in  existing  instructional 
technologies.  Although  studies  on  the  effectiveness  of  CBT  indicate  that  it  is  capable  of  meet- 


Multimedia  is  defined  here  as  digital  audio,  video,  images,  music,  text,  or  animation  that  is  integrated  on  a  com¬ 
puter  system. 
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ing  basic  training  requirements,  it  is  more  limited  than  full-function  multimedia  on  educational 
grounds. 

Costs 

Training  developed  for  CBT  will  have  to  be  developed  according  to  the  specific  needs  of  the 
organization.  This  development  implies  creating  the  software  in  conjunction  with  a  developer 
and  the  management  of  all  the  activities  thereof. 

5.3.2  Strategy  2:  Combined  Semi-Interactive  Instruction 

The  combined  semi-interactive  strategy  is  designed  primarily  to  reduce  travel  costs  without  a 
loss  of  instructional  effectiveness. 

Objective 

Map  current  practice  into  a  combined  satellite-  and  computer-based  instructional  solution. 

Features 

This  approach  would  utilize  any  existing  satellite  infrastructure  in  combination  with  network- 
distributed  exercises  and  tests.  For  any  organizationwide  distributed  system  for  delivering 
courses,  an  advanced  record-keeping  system  will  have  to  be  created. 

Advantages 

This  approach  is  optimized  to  take  advantage  of  existing  resources.  The  only  new  program¬ 
ming  required  would  be  related  to  the  network-distributed  aspects  of  the  approach,  which 
would  provide  students  with  simple  text-based  exercises  and  tests  in  which  results  would  be 
stored  in  a  central  file  server. 

Limitations 

Although  interactive  video  communications  by  satellite  and  computer  network  are  improving, 
neither  is  easy  to  use.  Frame-display  rates  for  two-way  satellite  often  make  meetings  with  only 
a  few  individuals  somewhat  confusing.  This  becomes  even  more  problematic  in  scaling  to  full 
course  implementations.  Existing  courses  will  require  modifications  for  use  in  this  new  scenar¬ 
io  as  well.  Costs  are  high  for  delivery. 

Cost 

Costs  associated  with  this  approach  include  the  uplink  time,  course  test  and  exercise  devel¬ 
opment  and  design,  programming  for  those  materials,  and  lecturer  preparation  time. 

5.3.3  Strategy  3:  Computer-Based  Multimedia 

Currently  all  features  of  any  multimedia  system  can  be  used  on  LANs  linked  to  a  central  server 
or  as  stand-alone  systems.  Full-motion  video,  sound,  animation,  music,  and  a  high  level  of  in¬ 
teractivity  are  all  available  at  relatively  low  cost. 
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Objective 

Create  an  infrastructure  to  support  future  network-based  WWW  access  and  multimedia-based 
course  development. 

Features 

This  delivery  configuration  will  suit  any  scale  of  multimedia  delivery.  All  the  attributes  of  multi- 
media  can  be  delivered  to  student  workstations  on  LANs  or  on  stand-alone  units.  To  support 
the  capabilities  of  the  WWW,  it  is  necessary  for  the  target  workstations  to  have  full  multimedia 
capability. 

Advantages 

Multimedia  provides  effective  instruction.  By  virtue  of  the  range  of  possible  presentation  styles 
and  devices,  multimedia  provides  a  learning  environment  with  possibilities  previously  un¬ 
known  in  the  history  of  education.  With  proper  development,  interactivity  is  built  into  course 
delivery,  and  the  richness  and  variety  of  the  learning  environment  is  enhanced  by  photograph¬ 
ic-quality  images,  full-motion  video,  sound,  and  music. 

Limitations 

Initial  multimedia  course  development  is  expensive,  requires  management  and  supen/ision, 
and  requires  relatively  long  lead  times  to  create  a  successful  system.  Further,  poor  multimedia 
is  always  a  potentially  costly  risk.  Developing  multimedia  programs  from  scratch  is  a  complex 
process  that  requires  careful  expert  supervision. 

Cost 

As  stated  above,  multimedia  has  many  initial  costs  associated  with  it.  These  costs  are  prima¬ 
rily  incurred  as  functions  of  course  and  project  planning,  creation  of  graphics,  animations,  and 
screen  design.  Video  used  in  multimedia  is  no  more  or  less  expensive  than  in  any  other  video 
production.  Programming  time  is  somewhat  more  extensive  than  with  a  CBT  program,  al¬ 
though  the  nature  of  the  work  is  not  substantially  different.  If  development  capability  is  not  cre¬ 
ated  within  the  organization,  there  will  be  costs  incurred  to  supervise  a  developing 
subcontractor.  Building  the  capability  in  house  is  difficult,  with  a  steep  learning  curve. 

5.3.4  Strategy  4:  Real-Time  Internet-Based  Instruction 

The  Internet  is  perhaps  the  fastest  growing  educational  technology  in  existence.  According  to 
statistics  gathered  at  Software  Engineering  Institute  (SEI),  the  Internet  is  growing  at  a  rate  of 
7  to  10  percent  per  month.  Corresponding  to  this  growth  rate  are  the  tools  for  navigating  and 
communicating  by  the  network.  Carnegie  Mellon  University  is  currently  producing  a  computer- 
based  tool  for  collaborative  distance  work.  Individuals  may  see  one  another  by  video  as  well 
as  being  able  to  write  and  critique  documents  in  real  time.  While  this  technology  requires  that 
each  workstation  have  its  own  video  card,  it  is  only  a  matter  of  time  before  this  type  of  system 
is  commonplace  for  office  work,  and  there  is  no  factor  in  particular  that  would  prevent  its  use 
for  classroom  interaction. 
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Objective 

Provide  an  easy-to-use  Internet  front  end  for  interactive  teacher-student  dialogue. 

Features 

Collaborative  work  systems  are  not  new;  however,  collaborative  writing  tools  that  include  vid¬ 
eo  are  only  now  becoming  available  for  multichannel  communications  on  the  Internet.  With 
multi-party  communications,  an  instructor  and  class  can  participate  in  dialogue  from  multiple 
distant  locations.  Various  programs  currently  exist  to  support  this  kind  of  interaction,  and  the 
technology  to  facilitate  this  type  of  network-based  dialogue  is  rapidly  improving. 

Advantages 

Relative  to  other  computer-based  distance-learning  options,  this  option  can  be  made  available 
with  less  effort.  No  new  materials  would  have  to  be  developed  to  conduct  a  class.  In  this 
sense,  the  Internet  is  rapidly  becoming  like  two-way  satellite  communications.  Materials  de¬ 
veloped  specifically  for  this  medium  require  some  modification  to  optimize  on  the  dialoguing 
aspect,  but  no  special-purpose  materials  such  as  graphics,  animations,  or  screen  designs 
would  have  to  be  created. 

Limitations 

There  will  be  hardware  required  to  make  full-duplex  video  and  audio  communications  possi¬ 
ble.  If  the  implementation  platform  is  Intel  based,  video  frame  rates  may  be  fewer  than  15 
frames  per  second.  Video  display  on  computer  terminals  will  in  all  likelihood  be  less  than  full 
screen  for  at  least  one  year.  Currently  the  size  of  these  displays  is  either  320  x  240  (quarter 
screen)  or  160  x  120  (one  eighth).  Naturally  the  size  of  the  video  display  window  is  a  limitation 
relative  to  any  workstation’s  monitor  size. 

Cost 

The  cost  for  this  solution  will  stem  primarily  from  hardware  purchases  and  network  installation. 
Each  unit  would  require  a  video  card  plus  enough  memory  and  CPU  to  provide  students  with 
equivalent  interactive  capability. 

5.4  Conclusions 

The  rapid  rise  of  the  Internet  and  computer-based  multimedia  have  opened  the  door  to  edu¬ 
cational  delivery  possibilities  that  have  never  been  see  before.  Students  of  all  ages  can  now 
have  access  to  video  archives  of  lunar  travel,  photos  of  the  Shoemaker-Levy  Comet  as  it 
crashed  into  Jupiter,  international  communications,  and  archives  describing  foreign  peoples 
in  full  color,  with  music  and  voice.  These  technologies  are  no  longer  restricted  to  elite  research 
organizations.  Soon  it  is  likely  that  they  will  be  in  homes  all  across  the  United  States.  The  na¬ 
ture  of  training  and  education  will  be  changed  forever  as  a  result.  Organizations  are  no  longer 
asking  questions  that  begin  with  “If  we  get  on  the  Internet...”  but  rather  they  assert,  “When  we 
get  on  the  Internet...”  Because  of  the  intense  demand  of  this  user  population,  the  very  nature 
of  learning  will  change.  The  demand  will  be  on  improved  interfaces  and  more  interesting  and 
dynamic  computer-based  experiences. 
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This  report  has  only  touched  upon  satellite  instruction.  No  matter  which  final  distance-learning 
strategy  is  selected,  it  appears  that  satellite  instruction  is  always  a  possibility,  given  the  ready 
resource  that  it  is.  The  cost  for  satellite  delivery  appears  to  remain  relatively  constant  and  is 
expensive.  In  my  estimation,  satellite-  and  computer-based  delivery  solutions  will  be  compet¬ 
itive  in  the  short  term;  however,  the  cost  of  satellite  delivery  will  remain  high  while  the  cost  of 
network-based  solutions  allow  for  reduction  through  amortization  of  their  initial  development 
cost. 

The  military  has  in  many  instances  led  the  development  of  leading-edge  educational  technol¬ 
ogy,  and  should  continue  to  do  so.  To  pursue  a  comprehensive  strategy  to  employ  these  tech¬ 
nologies  effectively,  however,  sufficient  resources  must  be  committed  to  provide  not  only 
adequate  materials,  but  human  energy  as  well.  The  change  to  network-based  multimedia  is 
inevitable.  The  questions  that  remain  pertain  only  to  the  logistics  of  the  implementation. 
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Appendix  A  Course  Example 

This  appendix  describes  a  protocol  to  assist  in  selecting  from  among  instructional  delivery 
strategies. 

A.1  The  Questions  to  Ask 

With  any  project  development  where  training  may  be  an  issue,  question  number  1  is  “Do  I 
have  an  instructional  problem  to  solve?”  The  question  can  have  subtle  implications — ^to  an¬ 
swer  it,  one  must  make  a  delineation  between  instruction  and  the  providing  of  information.  In 
terms  of  this  report,  instruction  is  defined  as  a  set  of  activities  designed  to  impart  verifiable 
changes  in  student’s  cognitive  or  procedural  abilities  toward  specific  learning  objectives.  By 
contrast,  providing  information  is  a  communication  designed  as  a  one-way  transmission  of 
ideas.  In  other  words,  providing  information  requires  no  verification  of  its  effectiveness  in  ac¬ 
complishing  the  goal,  whereas  in  this  context,  instruction  does. 

Once  the  decision  is  made  regarding  instruction  vs.  providing  information,  it  becomes  appro¬ 
priate  to  ask  whether  technology  might  provide  some  benefit  to  the  course  delivery.  Because 
current  technologies  are  proven  to  be  of  benefit  in  many  instructional  contexts,  it  is  not  prema¬ 
ture  to  begin  examining  this  question  before  content  development,  because  many  times  there 
is  a  purely  logistical  component  to  the  technology  question.  For  example,  if  students  are  widely 
dispersed  geographically  and  instructor  resources  are  low,  some  form  of  distance-learning 
technology  should  be  considered. 

The  next  set  of  questions  to  be  addressed  revolves  around  content  development.  Instructional 
systems  design  (ISD),  as  defined  by  various  authors,^  ^  assumes  a  fully  integrated  approach 
to  curriculum  development  that  includes  the  following  steps  at  a  high  level: 

1 .  Identify  instructional  goals. 

2.  Conduct  instructional  analysis. 

3.  Identify  (student)  entry  behaviors. 

4.  Write  performance  objectives. 

5.  Develop  criterion-referenced  test  items. 

6.  Develop  instructional  strategy. 

7.  Develop  and  select  instructional  materials. 

8.  Design  and  conduct  formative  evaluation. 

9.  Design  and  conduct  summative  evaluation. 


Prominent  among  these  are  Walter  Dick  and  Lou  Carey  of  Florida  State  University  [Dick  85]. 


CMU/SEI-95-TR-004 


39 


While  the  precise  implementation  of  these  steps  may  change  from  one  instructional  designer 
to  another,  ISD  is  based  on  a  set  of  learning  principles  that  have  become  commonplace 
among  instructional-design  professionals. 

While  it  is  beyond  the  scope  of  this  report  to  provide  an  exhaustive  discourse  on  each  part  of 
the  ISD  model,  there  are  assumptions  of  the  model  that  have  a  direct  bearing  on  the  instruc¬ 
tional  selection  process.  For  example,  ISD  always  assumes  that  a  set  of  learning  objectives 
must  be  established.  At  first,  this  often  amounts  to  identifying  “key  ideas,”  or  what  otherwise 
might  be  called  the  conceptual  core  of  the  curriculum.  Broadly  speaking,  the  question  in  this 
phase  of  development  is  “What  set  of  understandings  do  I  want  my  students  to  have  once  they 
have  finished  with  the  curriculum?”  Often  a  curriculum  committee  or  other  team  brainstorms 
these  ideas.  At  first  they  may  be  identified  as  simple  two-  or  three  -word  labels,  such  as  partial 
differential  equations,  linear  algebra,  key  signature,  color  mixing,  etc.  In  other  words,  in  this 
part  of  the  curriculum  development  a  high-level  set  of  ideas  within  a  domain  is  articulated. 

Once  the  set  of  ideas  is  elaborated  satisfactorily,  relationships  between  the  ideas  should  be 
determined.  This  phase  assumes  that,  in  general,  students  learn  best  when  ideas  are  present¬ 
ed  in  a  simple  form  at  first,  with  more  complexity  introduced  as  this  knowledge  is  integrated 
with  new  information  and  varied  contexts.  As  such,  the  ISD  model  advocates  organizing  a  map 
of  the  concepts  to  show  these  relationships  explicitly.  For  example,  in  Figure  4  on  page  41 , 
the  high-level  concepts  are  organized  from  simpler  to  more  complex.  Instructors  might  argue 
about  which  ideas  are  more  difficult  and  which  ones  belong  first  in  the  curriculum,  but  the  value 
of  the  visual  representation  is  not  in  question.  Further,  the  “concept  map”  can  be  augmented 
to  show  how  the  ideas  are  related  to  one  another  in  the  horizontal  direction  as  well.  In  Figure 
4,  the  relationship  of  variable  assignment  in  structures  needs  to  be  accounted  for. 

Next,  for  each  of  the  concepts  outlined,  behavioral  objectives  must  be  defined.  An  assumption 
of  the  ISD  model  is  that  both  physical  and  cognitive  actions  can  be  assessed  only  by  using 
behavioral  indicators. Therefore,  the  curriculum  developer  must  identify  the  exact  nature  of 
the  student’s  desired  performance  for  any  level  of  knowledge. 


Hence  Bloom’s  Taxonomy  [Bloom  56]  is  entitled,  “Taxonomy  of  Educational  Objectives:  Book  1 ,  Cognitive  Do¬ 
main.” 
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Terminal  Concept 


•  ••etc. 


etc. 


Figure  4:  Concept  Map 


A.2  Example  for  a  Specific  Course 

As  a  first  example,  consider  critical  competency  1  shown  in  Figure  5.  This  critical  competency 
is  focused  on  Bloom’s  level  1  as  the  target  cognitive  implementation  level.  The  objectives  for 
this  competency  are  also  listed  in  Figure  5.  To  create  a  more  detailed  picture  of  the  classroom 
implementation  of  the  critical  competency  and  how  it  relates  to  Bloom,  there  must  be  a  precise 
description  of  the  activities  carried  out  by  the  students. 
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1.  Risk  Management 


Critical  competency  1 :  to  identify  typical  software-acquisition  risks  for  systems, 
select  appropriate  risk-mitigation  strategies,  and  evaluate  the  relative  merits  of 
those  strategies 

Top-level  enabler  1:  the  ability  to  evaluate  acquisition  programs,  identify  risks, 
identify  associated  risk-mitigation  approaches,  and  adjust  acquisition  plans  to 
incorporate  risk-mitigation  approaches 

Enabling  knowledge 

1 K1 :  concept  of  “risk” 

1 K2:  risk-identification  processes 

1 K3:  risk-assessment  processes 

1 K4:  risk-reduction  strategies 

1 K5;  problem-solving  techniques. 

1 K6:  laws,  regulations,  and  DoD  policies  pertaining  to  the  acquisition  of 

software- intensive  weapon  systems;  command,  control,  communications, 
and  intelligence  (C3I)  systems;  and  automated  information  systems  (AISs) 
1 K7:  common  software-acquisition  risks 
1 K8:  role  of  working  groups  in  risk  management 

1 K9:  software  development  processes 

1 K1 0:  risk-management  organization 

Enabling  skills 

1 S1 :  identifying  sources  and  types  of  risks  in  software-intensive  system 
acquisitions 

1 S2;  identifying  the  causes  of  known  software-acquisition  risks 
1  S3:  establishing  and  applying  a  process  for  recognition  and  awareness  of 
software-acquisition  risk 

1 S4:  evaluating  contractor  risk-management  processes  to  determine  the  extent 

to  which  they  are  consistent  with  and  support  the  program  risk- 
management  strategy 

1 S5:  predicting  the  effect  and  consequence  of  a  risk  event 

1 S6:  estimating  the  probability  of  a  software-acquisition  risk  event 

1S7:  deveioping  software-acquisition  risk-reduction  strategy 

1 S8:  evaiuating  software-acquisition  problem-solving  techniques  to  determine 

the  extent  of  their  effectiveness  in  reducing  program  risk 

Figure  5:  Sample  Objectives  for  a  Course  in  Risk  Management 


Given  the  “enabling  knowledge”  provided  from  items  1 K1  through  1 K1 0,  what  will  be  the  exact 
actions  and  success  criteria  expected  of  students  when  they  carry  out  enabling  skill  1S1:  iden¬ 
tifying  sources  and  types  of  risks  in  software-intensive  system  acquisitions?  We  can  assume  that 
there  is  more  than  one  way  of  assessing  software  acquisition  risks.  Is  there  more  than  one 
way  of  assessing  student  proficiency  at  doing  so?  Therefore,  the  exact  nature  of  an  in-class 
exercise  must  be  specified,  not  only  to  meet  the  requirements  of  the  ISD  model,  but  in  the  case 
of  the  use  of  interactive  technologies,  to  provide  information  to  developers. 
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A.2.1  Technology  and  a  Classroom  Example 

A  further  breakdown  of  knowledge  requires  the  following  further  clarification: 


Learning  Objectives  1S1K1L 

The  student  shall  list  four  sources  of  software-intensive  system  cost-estimation 
risk  and  for  each  source  identify  the  a)  associated  possible  risk  events,  b) 
probability  of  those  risk  events,  and  c)  possible  consequences  of  the 
occurrence  of  each  risk  event. 

The  instructional  method  listed  for  this  objective  is:  “class  discussion  followed  by  student  par¬ 
ticipation  in  providing  4  or  5  examples  to  complete  the  following  outline  on  the  white  board.” 
The  “white  board”  example  is  as  follows: 


Source 

Possible  Risk  Event 

Probability 

Possible  Consequences 

Ambiguous 
statement  of 
work 

Cost  estimation  low 
by  a  factor  of  4 

High 

Contractor  exhausts  manage¬ 
ment  reserve. 

Reduced  capability  in  delivery 
system. 

Late  initial  operational  capability 
Program  manager  relieved 

Table  A-1 :  Cost-Estimation  Risks 


A.2.2  Samples  of  Instructional  Technology  Application 

Given  the  requirements  specified  above,  a  learning  technology  developer  must  ask  several 
questions: 

1 .  Given  the  nature  of  the  classroom  interaction,  is  there  a  practical  role  for 
learning  technologies  in  this  scenario? 

2.  If  technology  is  to  be  used,  how  will  it  serve  the  content? 

3.  Which  technology  best  serves  the  need? 

Assuming  that  we  determine  that  some  form  of  instructional  technology  will  be  used,  we  begin 
to  look  for  answers  to  question  number  2.  To  use  any  form  of  instructional  technology,  we  must 
at  the  outset  determine  its  role  from  the  standpoint  of  logistics.  Will  it  assist  the  instructor,  or 
do  we  actually  wish  to  offload  portions  of  the  overall  curriculum  entirely? 

A.2.2.1  Satellite 

As  stated  earlier,  satellite  offers  many  of  the  features  of  a  normal  face-to-face  interaction  with 
an  instructor,  depending  on  the  capability  of  the  satellite  system.  Satellite  capability  has  a 
range  that  permits  full  two-way  audio  and  video  potential,  one-way  video/two-way  audio,  or 
just  one-way  video  and  audio.  In  almost  any  current  classroom  use  of  satellite,  the  most  likely 
configuration  will  be  one-way  video/two-way  audio,  because  of  the  expense  of  full-duplex  sat¬ 
ellite  and  the  inadequacy  of  one-way  usage. 
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With  either  of  the  interactive  satellite  configurations,  the  interaction  between  instructor  and 
class  could  be  carried  out  more  or  less  in  a  fashion  like  classroom  lecture  and  discussion.  Be¬ 
cause  the  instructor  is  separated  by  distance  and  will  in  most  cases  have  no  visibility  into  the 
classroom,  this  presents  some  obvious  limitations.  For  example,  in  the  likely  case  of  one-way 
video/two-way  audio,  the  instructor  will  have  to  keep  a  list  of  the  students’  names  to  refer  to 
during  instruction.  Obviously  this  is  less  conducive  of  interaction  than  line-of-sight  visibility. 

A.2.2.2  Interactive  Multimedia-Based  Solutions 

Interactive  multimedia  opens  the  door  to  many  options  for  the  classroom.  The  following  para¬ 
graphs  describe  potential  levels  of  multimedia  implementation  for  classroom  support. 

Classroom  Support 

For  instructors  who  are  not  particularly  comfortable  with  group  dynamics,  performance  sup¬ 
port  can  be  engineered  to  assist  the  teacher  in  real  time.  One  relatively  simple  use  of  this  tech¬ 
nology  is  as  a  video  repository.  As  an  example,  the  instructor  could  select  a  video  segment 
showing  a  form  of  environmental  hazard,  followed  by  a  question  to  the  class:  “What  is  the 
proper  procedure  for  disposing  of  this  class  A  hazard?’’  Students  and  instructor  would  then  dis¬ 
cuss  a  solution.  The  instructor  could  then  demonstrate  the  effects  of  the  students’  selections. 

Using  interactive  multimedia  in  this  way  is  no  different  from  using  videotape  as  classroom  sup¬ 
port  with  the  following  important  exception:  The  computer  permits  real-time  classroom  inter¬ 
action  because  of  its  rapid  random-access  capability.  Videotape  is  simply  too  slow.  If  the 
instructor  were  to  try  to  use  a  standard  videotape  to  recreate  this  level  of  interactivity,  not  only 
would  all  the  appropriate  tracks  have  to  be  indexed,  but  the  instructor  would  be  left  to  find  them 
all  using  the  fast-forward  button  on  the  VCR— the  class  waits  as  the  VCR  scrolls  tape.  Using 
the  computer’s  rapid  access  time  and  database  features,  the  instructor  could  predefine  paths 
through  video  segments  that  contain  useful  illustrations  of  concepts  that  would  otherwise  be 
difficult  to  explain  or  demonstrate. 

In  this  example,  one  of  the  procedures  selected  by  students  could  easily  cause  (in  video)  an 
explosion,  acid  spill,  contamination  of  the  water  table,  and  so  on— experiences  no  one  wishes 
to  see  played  out  in  life.  This  kind  of  dramatic  example  provides  rich  opportunities  for  further 
discussion  and  debate  in  class.  To  those  with  experience  in  videodisc  or  video  instruction,  this 
kind  of  classroom  multimedia  is  not  new,  but  it  is  effective,  and  the  random-access  capability 
permitted  by  the  computer  makes  using  video  in  this  way  practical  and  easy. 

This  basic  form  of  classroom  assistance  using  interactive  multimedia  is  among  the  least  com¬ 
plex  and  least  costly  to  produce.  Nonetheless,  as  with  all  multimedia,  there  will  be  greater  ini¬ 
tial  expense  than  with  traditional  classroom  approaches.  As  with  any  course  development 
effort,  however,  multimedia  development  should  begin  with  an  initial  assessment  of  the  needs 
and  requirements  of  the  system  in  which  the  proposed  instruction  is  to  be  used.  This  analysis 
is  called  a  front-end  analysis  or  needs  assessment. 

Using  the  sample  course  objective  on  page  43,  the  emphasis  for  an  interactive  multimedia 
system  would  be  to  provide  a  resource  for  instructors  to  enliven  the  activity  of  populating  the 
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whiteboard  diagram.  For  example,  the  instructor  could  create  a  repository  to  play  out  various 
crisis  scenarios  related  to  poor  cost-estimation  practices.  These  could  be  illustrated  in  video 
or  by  using  photos,  illustrations,  animations,  or  any  other  combination  from  the  multimedia  ar¬ 
mamentarium.  These  scenarios  could  be  stored  on  either  a  compact  disc  or  computer  hard 
drive.  With  the  addition  of  a  user  interface,  the  instructor  could  provide  the  class  with  cost-es¬ 
timation  scenarios  and  ask  for  student  opinions  about  “What  would  happen  in  the  case  where 
I  would  have  ‘ambiguous  statements  of  work’?  What  are  some  of  the  problems  I  might  encoun¬ 
ter?”  Working  together  to  create  a  usable  interface  and  to  capture  the  appropriate  variables  in 
the  cost  estimation  domain,  the  developer  and  instructor  could  create  a  repository  of  powerful 
examples  that  could  be  made  available  to  teachers  and  students  on  demand. 

In  the  case  of  cost  estimation,  instead  of  merely  populating  a  whiteboard  with  appropriate  en¬ 
tries,  the  class  would  now  be  able  to  propose  strategies  and  witness  some  of  the  unfortunate 
consequences  associated  with  poor  cost  estimation,  and  to  discuss  them  with  an  expert  in¬ 
structor  in  class,  thereby  avoiding  crises  in  real  life.  This  approach  is  strong  from  the  educa¬ 
tional  point  of  view,  because  the  more  realistic  the  instructional  experience,  the  more  likely  it 
is  to  be  remembered. 

The  main  limitation  to  creating  multimedia  tools  is  the  expense  of  producing  specialized  video, 
images,  and  animations,  as  well  as  the  accompanying  programming.  Clearly,  any  analysis 
preceding  this  form  of  development  has  to  consider  the  shelf  life  of  the  educational  product 
and  the  nature  of  the  materials  being  produced:  How  generic  are  they? 

A  classroom-support  video  repository  such  as  the  repository  I  have  described  would  allow  the 
possibility  of  materials  being  shared  and  reused,  as  well  as  the  creation  of  a  generic  interface 
that  could  allow  instructors  to  define  their  own  specialized  load-and-play  curricula. 

The  question  instructors  should  ask  themselves  in  approaching  interactive  multimedia  is 
“What  would  instantaneous  access  to  graphics,  sound,  and  video  do  for  me  in  my  classroom?” 

Instructors  are  often  their  own  best  technical  engineers.  A  recent  article  in  Multimedia  Today 
[IBM  95]  describes  Dr.  Jack  Wilson’s  use  of  multimedia  in  his  physics  class  at  Rensselaer 
Polytechnic.  Dr.  Wilson  describes  the  following  scenario  utilizing  various  technological  ele¬ 
ments  in  the  classroom: 

During  our  acceieration-due-to-gravity  session,  we  videotaped  a  student 
throwing  a  ball.  We  digitized  the  video  directly  into  the  computer  and  made  it 
available  to  every  student  workstation  over  the  network.  Students  then 
analyzed  the  motion  using  the  online  scientific  tools.  Next,  students  created  a 
spreadsheet  illustrating  the  ball’s  position  versus  time  data,  and  then  plotted 
their  results  on  computerized  graphs.  All  in  one,  two-hour  block. 

Joe  Molino,  the  IBM  consultant  with  Rennselaer,  is  quoted  as  saying,  “Rather  than  separating 
the  functions  of  lecture,  recitation  and  laboratory,  the  new  physics  format  allows  instructors  to 
move  freely  from  one  mode  to  another”  [IBM  95]. 
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To  create  multimedia  tools  to  assist  in  the  classroom  presentation,  decisions  must  be  made 
about  the  kinds  of  illustrations  that  would  enhance  the  students’  grasp  of  the  content.  As  sug¬ 
gested,  the  best  examples  are  drawn  from  real  life.  Such  decisions  are  most  often  made  at  a 
creative  meeting  between  the  developer  of  the  multimedia  tool  and  the  content  developer. 

The  Just-in-Time  (JIT)  Approach  to  Interactive  Multimedia 

Using  the  JIT  approach  would  simply  entail  recording  the  lectures  of  the  instructors  with  a  vid¬ 
eo  camera,  digitizing  them,  and  indexing  their  content  according  to  a  topic  outline  prepared  by 
the  instructor.  The  lecture  and  indexes  would  then  be  incorporated  into  a  user  environment 
that  permits  electronic  mail  to  the  instructor  (or  assistant)  so  the  students  could  ask  questions. 
The  system  performs  a  first-pass  analysis  on  the  student’s  question  to  determine  whether  it 
has  been  asked  and  answered  in  the  past  using  certain  natural-language  analysis  techniques. 
If  the  question  has  been  asked  in  the  past,  the  system  answers  it  automatically,  using  the  an¬ 
swer  already  stored  in  its  database  of  answers  created  by  the  instructor.  If  the  question  has 
not  been  asked  before  (or  the  system  cannot  analyze  it  accurately),  the  system  passes  it  along 
to  the  instructor  to  answer  in  electronic  mail. 

Of  course  this  method  does  not  provide  for  any  real-time  classroom  dialogue  with  the  instruc¬ 
tor,  and  therefore  lacks  the  benefit  of  these  dynamics.  Relative  to  more  elaborate  uses  of  mul¬ 
timedia,  this  approach  is  inexpensive  to  produce,  allows  for  network  connectivity,  provides  a 
direct  link  to  the  instructor  by  electronic  mail,  and  a  provides  a  consistent  reference  for  course 
information  through  the  stored  lecture. 

Stand-Alone  Interactive  Multimedia  Systems 

Any  stand-alone,  computer-based  instruction  derives  its  strength  from  the  curriculum  and  the 
richness  of  the  intellectual  content.  When  a  stand-alone  system  is  created,  the  effort  must  be 
a  collaboration  between  the  programmer,  graphics  specialist,  instructional  designer,  project 
manager,  and  primary  content  developers.  To  be  effective,  stand-alone  systems  must  be  en¬ 
gaging  and  task  intensive.  These  systems  are  used  to  best  effect  when  they  simulate  events 
and  ideas  that  are  difficult  to  express  in  words  alone.  To  achieve  a  high  level  of  effectiveness 
using  this  medium,  there  must  be  an  exacting  analysis  of  the  conceptual  content  of  the  course, 
planning  of  the  illustrations  of  the  tasks  and  presentations,  and  creative  brainstorming  to  de¬ 
termine  an  overall  approach  to  teaching  the  curriculum.  This  effort  requires  a  complete  trans¬ 
lation  of  the  curriculum  from  a  facilitated  discussion  or  lecture  into  a  wholly  different  paradigm. 

Given  the  example  provided  on  page  42,  there  would  be  a  need  to  determine  how  best  to  get 
across  ideas  at  the  level  of  the  “top-level  enabler.”  The  question  to  ask  for  enabler  number  1 
would  be  “What  presentations  and  activities  will  best  support  the  student’s  understanding  of 
risk  management?”  At  first,  there  is  a  need  to  know  whether  risk  management  has  subparts. 
Are  there  other  enablers  beneath  risk  management  that  constitute  prerequisite  understand¬ 
ings?  If  we  assume  that  risk  management  is  the  fundamental  concept  for  the  student  to  grasp, 
then  the  developers  must  work  in  conjunction  with  the  content  experts  to  determine  an  ap¬ 
proach  to  teaching  it. 
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For  example,  one  approach  might  use  video  segments,  graphics,  animations,  or  audio  to 
present  a  simulated  environment.  One  such  system,  produced  by  Allen  Communications, 
places  the  student  in  the  position  of  managing  a  railroad.  The  learner  must  make  decisions 
about  real-estate  purchases,  right  of  way,  legal  considerations,  budget  decisions,  competitors, 
and  so  on.  The  idea  behind  the  system  is  to  present  the  student  with  a  realistic  representation 
of  the  variables  affecting  a  manager  in  real  life,  using  an  entertaining  backdrop.  In  this  simu¬ 
lation,  students  sometimes  achieve  great  success  and  sometimes  go  bankrupt,  depending  on 
their  judgments  about  new  rail  purchases  or  construction,  contracts  obtained,  and  so  on. 

For  the  risk  management  example,  a  series  of  video  scenarios  could  be  compiled  illustrating 
the  effects  of  certain  variables  and  risk.  The  possibilities  for  stand-alone  interactive  multimedia 
systems  are  many,  given  the  following  set  of  enabling  skills: 

IS1 :  identifying  sources  and  types  of  risks 

IS2:  identifying  the  known  causes  of  software-acquisition  risks 

IS3:  establishing  and  applying  a  software  risk  recognition  and  awareness 
process 

IS4:  predicting  the  consequences  of  a  software-acquisition  risk  event 

IS5:  estimating  the  probability  of  a  software  risk-acquisition  risk  event 

IS6:  developing  risk-reduction  strategies  for  software  acquisition 

IS7:  evaluating  software-acquisition  problem-solving  techniques 

Having  identified  the  enabling  skills,  the  next  question  to  ask,  according  to  the  instructional 
systems  design  model  [Dick  85],  is  “What  instructional  strategy  will  be  used?”  Using  interactive 
computer-based  multimedia,  the  instructor  has  a  new  set  of  possibilities  available.  Given  the 
enabling  skills  above,  the  instructor  attempts  to  capture  the  best  way  to  “situate”  the  learner’s 
experience  such  that  it  will  have  the  greatest  meaning  and  transfer  of  ability  into  the  actual 
work  context.  Obviously,  one  way  of  accomplishing  this  is  to  simulate  an  actual  acquisition  en¬ 
vironment.  The  task  would  be  to  create  a  scenario  in  which  the  learner  would  be  put  into  a 
context,  perhaps  using  video  to  accomplish  this,  and  be  forced  to  make  decisions  according 
to  the  risks  that  arise. 

This  approach  creates  a  surrogate  experience,  allowing  the  user  to  make  mistakes  and  see 
the  consequences  of  those  mistakes  played  out  in  a  form  that  causes  no  disasters,  but  which 
is  memorable  to  the  student. 

Consider  the  following  scenario: 

A  student  takes  a  seat  in  front  of  a  terminal  in  an  office.  The  system  is  running  from  a  LAN, 
and  all  that  is  required  of  the  student  to  begin  instruction  is  to  type  a  user  identification  and 
password.  After  the  student  logs  in,  the  system  invites  the  student  to  answer  some  questions 
concerning  organizational  affiliation,  branch  of  service,  type  of  assignment,  and  so  on.  After 
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conducting  this  interactive  questionnaire,  the  system  loads  a  scenario  appropriate  to  the  us¬ 
er’s  assignment  and  branch  of  service,  based  on  pre-established  curriculum  requirements. 

For  this  example,  the  student  is  assigned  as  a  captain  in  the  systems  program  office  (SPO) 
for  COBRA  STARS,  a  fictitious  airborne  surveillance  aircraft.  Colonel  Brighton  is  the  program 
director,  and  the  user  will  be  assigned  as  a  software  division  chief.  In  this  role,  the  captain  will 
be  approached,  on  video,  by  a  Mr.  Johnson  from  THC  Systems  Inc.  THC  was  tasked  with  pro¬ 
ducing  a  radar  data  processor  (RDP)  for  COBRA  STARS.  The  RDP  is  actually  an  integrated 
set  of  processors,  each  with  a  specific  function  relative  to  the  signal-processing  and  decoding 
task,  each  processor  having  its  own  specialized  software  modules  to  drive  it.  THC  named 
these  individual  processors  AMPS. 

In  the  original  software  requirements  specification  (SRS)  for  the  system,  all  processors  were 
to  have  50  percent  of  capacity  idle  at  all  times  during  system  use.  Today  there  is  a  problem: 
The  captain  notices  that  in  many  instances,  performance  measures  of  the  processors  in  the 
RDP  are  functioning  at  levels  well  over  50  percent.  At  the  same  time,  Mr.  Johnson  assures  the 
captain  that  the  RDP  is  running  within  the  specifications  provided  to  THC  by  the  Air  Force. 

In  video.  Colonel  Brighton  angrily  demands  that  the  captain  get  to  the  bottom  of  this  problem, 
as  he  is  convinced  that  the  contractor  is  attempting  to  renegotiate  the  original  specifications. 

After  this  presentation  of  the  conditions  within  the  SPO,  the  captain  (student  user)  is  shown  a 
menu  of  options  representing  possible  courses  to  be  taken  given  these  circumstances.  Among 
the  possibilities  are 

•  request  for  specifications  for  the  RDP 

•  performance  measures 

The  student  may  choose  one  or  both  of  the  options  listed.  Depending  on  whether  one  or  both 
have  been  selected,  the  student  is  approached  by  an  animated  figure  who  suggests  that  an¬ 
alyzing  the  original  performance  specifications  might  provide  some  useful  clues.  However,  if 
only  one  of  the  possibilities  is  chosen  in  the  absence  of  the  other,  the  student  is  simply  pre¬ 
sented  with  the  opportunity  to  schedule  a  meeting.  The  possibilities  given  the  student  for  this 
meeting  include 

•  Meet  with  Colonel  Brighton  to  discuss  specifications. 

•  Meet  with  the  contractor  to  discuss  the  specifications  and  performance 
measures. 

For  the  case  in  which  the  student  has  not  selected  the  opportunity  to  review  the  performance 
measures,  and  must  now  schedule  a  meeting,  the  captain  is  in  effect  walking  into  a  trap. 
Whether  the  meeting  takes  place  with  the  colonel  or  the  contractor,  questions  will  be  posed 
by  either  the  contractor  or  the  colonel  that  will  expose  the  captain’s  lack  of  preparation. 

For  the  case  in  which  “Meet  with  Colonel  Brighton”  has  been  selected,  the  colonel  (in  video) 
will  ask,  “What  do  the  comparisons  say?”  If  the  student  at  the  workstation  provides  no  answer 
within  a  given  timeout  threshold  or  selects  a  menu  option  such  as  “I  have  not  yet  requested 
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that  information,”  the  colonel  dismisses  the  captain  brusquely  with,  “At  least  next  time  prepare 
yourself  before  you  waste  my  time.  Captain!” 

Of  course,  other  options  would  provide  a  series  of  other  complex  experiences.  The  “right  an¬ 
swer,”  in  the  case  above,  would  be  to  have  chosen  both  the  performance  measures  and  spec¬ 
ifications,  which  in  turn  would  have  made  other  data-gathering  opportunities  available  to  the 
captain.  As  the  captain  made  more  decisions  pertaining  to  data  gathering,  options  such  as  “ex¬ 
amine  individual  RDP  component  performance  measures”  would  reveal  themselves  in  the 
context  of  the  captain’s  investigation,  with  subsequent  scenarios  and  opportunities  played  out 
in  video  or  animations,  simulating  the  consequences  of  making  those  decisions  as  realistically 
as  possible. 

Sidebar  (fyi);  In  the  true  story  on  which  the  above  example  was  based,  the  aggregated  perfor¬ 
mance  of  the  processors  within  the  RDP  left  more  than  50  percent  idle  capacity  for  the  RDP 
considered  as  a  whole.  The  government,  however,  asserted  that  each  processor  (AMP)  was 
to  have  50  percent  idle  capacity,  not  the  RDP.  The  software  runs  on  the  AMPs,  and  there  is 
no  high-order  language  or  compiler  for  the  RDP.  An  investigation  over  the  meaning  of  the  word 
“processor”  in  the  original  specification  ensued  because  of  the  government’s  assertion.  In  light 
of  the  government’s  actual  specification,  the  aggregated  performance  measures  used  by  the 
contractor  were  irrelevant. 

The  details  of  an  interactive  implementation  of  such  a  scenario  are  as  varied  as  one  can  imag¬ 
ine,  considering  the  many  possible  ways  of  utilizing  video,  audio,  animation,  and  so  on  to  cre¬ 
ate  a  poignant  instructional  experience  for  the  student.  Incorporating  enabling  skills  and 
knowledge  is  a  matter  of  specifying  a  mapping  between  the  types  of  tasks  assigned  to  the  stu¬ 
dent  and  those  enablers  in  the  context  of  the  multimedia  interaction.  In  the  simple  example 
above,  the  student  is  presented  with  a  scenario  in  which  choices  must  be  made  about  courses 
of  action  in  investigating  a  situation  in  which  risk  has  already  become  crisis.  By  being  thrust 
into  the  situation,  the  student  must  necessarily  “identify  sources  and  types  of  risks...,”  “estab¬ 
lish  and  apply  a  software  acquisition  and  awareness  process...”  and  so  on.  The  precision  and 
degree  with  which  the  course  enablers  are  fulfilled  depends  upon  the  creativity  and  analysis 
of  the  combined  team  of  content  and  multimedia  developers. 

A.2.3  Classroom  Technology  Decision  Points 

The  landscape  of  educational  decision  making  has  changed  with  advances  in  instructional 
technology.  If  the  ISD  model  (or  other  variation)  is  assumed  as  the  basis  for  course  develop¬ 
ment,  then  not  only  can  we  say  that  the  decisions  themselves  have  changed,  but  the  timing 
as  to  when  the  decisions  are  made  has  also  changed  in  reference  to  this  model. 

No  longer  must  one  ask  whether  instructional  technology  is  usable  in  the  classroom — it  is.  The 
question  to  ask  is  whether  one  prefers \o  use  technology  at  all  in  the  context  of  the  curriculum, 
and  if  so,  how?  In  terms  of  the  instructional-design  model  in  Figure  A-1  on  page  51 ,  this  means 

^3  Special  thanks  to  Joe  Besselman  and  Colonel  Thomas  Miller  for  providing  the  substance  of  this  example  from 
real  life. 
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that  it  is  now  possible  to  ask  whether  or  not  to  use  instructional  technology  at  the  same  time 
that  instructional  goals  are  considered,  as  opposed  to  waiting  until  we  reach  Develop  Instruc¬ 
tional  Strategy  or  Develop  and  Select  Instructional  Materials.  This  is  a  big  change  over  past 
practice,  in  which  the  use  of  instructional  technology  would  have  been  considered  only  well 
after  the  objectives  of  the  course  were  determined. 
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